Re: Secrets of Buffer Cache Enlargement.
2013/3/5 Bob Beck b...@openbsd.org: On Mon, Mar 4, 2013 at 4:24 PM, Bob Beck b...@openbsd.org wrote: You too can have a GIANT buffer cache etc. etc... Great.. and now I have people mailing me dmesg's from machines with 16 and 32 Gigs of ram. I only have 8 I feel so. small... My amd64 testbox running this has only 6G, so we can cry over red wine together.. 8-( the 1TB boxes at work have too many cores for obsd to run on them so I can't really make dmesg pr0n on them. The box randomizes bufmemcachepercent via cron every minute and has survived the night doing cvsyncs, cvs-up and racing over the ports tree back and forth. Since I don't have other pressure on it yet, my systat - dmapages is almost equal to total pages: (doing html mail here to get proportional font, sorry for that) 1 usersLoad 2.05 1.47 0.84 Wed Mar 6 09:23:22 2013 DEVICE READWRITE RTPSWTPS SECSTATS sd0 5898240 36 0 0.0 311384 total pages cd0000 0 0.0 311356 dma pages Totals5898240 36 0 0.0 36 dirty pages 9 delwri bufs 0 busymap bufs 6553 avail kvaslots 6553 kvaslots 0 pending writes 0 pending reads 2304 cache hits -- May the most significant bit of your life be positive.
Re: Secrets of Buffer Cache Enlargement.
Date: Wed, 6 Mar 2013 09:25:51 +0100 From: Janne Johansson icepic...@gmail.com 2013/3/5 Bob Beck b...@openbsd.org: On Mon, Mar 4, 2013 at 4:24 PM, Bob Beck b...@openbsd.org wrote: You too can have a GIANT buffer cache etc. etc... Great.. and now I have people mailing me dmesg's from machines with 16 and 32 Gigs of ram. I only have 8 I feel so. small... My amd64 testbox running this has only 6G, so we can cry over red wine together.. 8-( the 1TB boxes at work have too many cores for obsd to run on them so I can't really make dmesg pr0n on them. Even when you disable HyperThreading?
Re: Secrets of Buffer Cache Enlargement.
2013/3/6 Mark Kettenis mark.kette...@xs4all.nl My amd64 testbox running this has only 6G, so we can cry over red wine together.. 8-( the 1TB boxes at work have too many cores for obsd to run on them so I can't really make dmesg pr0n on them. Even when you disable HyperThreading? No, but when booting these machines, they take a ReallyLong(tm) time to figure out the size of 1TB ram even without strict tests, and I have only a few seconds to hit F2 for setup before they race onwards, so changing Bios setup to get down to only 40 cores for a super-quick test before someone starts missing the box is not easy. -- May the most significant bit of your life be positive.
ssl(8) cert generation instructions
It's not entirely obvious that -x509 actually means produce a csr, self-sign it (defaulting to SHA1), throw away the csr and write the cert and this had me stuck for a long time when I wanted to play with DSA server certs. So here's a diff which moves DSA cert generation instructions to the same style as RSA where the process is to produce a CSR and tell people how to sign it in separate steps. It doesn't take much longer and is clearer. As a bonus there are instructions for ECDSA cert generation. OK? Index: ssl.8 === RCS file: /cvs/src/share/man/man8/ssl.8,v retrieving revision 1.53 diff -u -p -r1.53 ssl.8 --- ssl.8 27 Nov 2012 01:02:07 - 1.53 +++ ssl.8 6 Mar 2013 12:53:51 - @@ -150,18 +150,66 @@ The following command will generate 1024 # openssl dsaparam 1024 -out dsa1024.pem .Ed .Pp -Once you have the DSA parameters generated, you can generate a certificate -and unencrypted private key using the command: +Once you have the DSA parameters generated, you can generate a Certificate +Signing Request and unencrypted private key using the command: .Bd -literal -offset indent -# openssl req -x509 -nodes -newkey dsa:dsa1024.pem \e - -out /etc/ssl/dsacert.pem -keyout /etc/ssl/private/dsakey.pem +# openssl req -nodes -newkey dsa:dsa1024.pem \e + -out /etc/ssl/dsacert.csr -keyout /etc/ssl/private/dsakey.pem .Ed .Pp To generate an encrypted private key, you would use: .Bd -literal -offset indent -# openssl req -x509 -newkey dsa:dsa1024.pem \e - -out /etc/ssl/dsacert.pem -keyout /etc/ssl/private/dsakey.pem +# openssl req -newkey dsa:dsa1024.pem \e + -out /etc/ssl/dsacert.csr -keyout /etc/ssl/private/dsakey.pem .Ed +.Pp +This +.Pa server.csr +file can then be given to a Certificate Authority who will sign the key. +.Pp +You can also sign the key yourself, using the command: +.Bd -literal -offset indent +# openssl x509 -sha256 -req -days 365 \e + -in /etc/ssl/private/dsacert.csr \e + -signkey /etc/ssl/private/dsacert.key \e + -out /etc/ssl/dsacert.crt +.Ed +.Pp +.Sh GENERATING ECDSA SERVER CERTIFICATES +First, generate parameters for ECDSA keys. +The following command will use a NIST/SECG curve over a 384 bit +prime field: +.Bd -literal -offset indent +# openssl ecparam -out ec-secp384r1.pem -name secp384r1 +.Ed +.Pp +Once you have the ECDSA parameters generated, you can generate a self-signed +certificate and unencrypted private key using the command: +.Bd -literal -offset indent +# openssl req -nodes -newkey ec:ec-secp384r1.pem \e + -keyout /etc/ssl/private/eccert.key -new \e + -out /etc/ssl/private/eccert.csr +.Ed +.Pp +To generate an encrypted private key, you would use: +.Bd -literal -offset indent +# openssl req -newkey ec:ec-secp384r1.pem \e + -keyout /etc/ssl/private/eccert.key -new \e + -out /etc/ssl/private/eccert.csr +.Ed +.Pp +This +.Pa eccert.csr +file can then be given to a Certificate Authority who will sign the key. +.Pp +You can also sign the key yourself, using the command: +.Bd -literal -offset indent +# openssl x509 -sha256 -req -days 365 \e + -in /etc/ssl/private/eccert.csr \e + -signkey /etc/ssl/private/eccert.key \e + -out /etc/ssl/eccert.crt +.Ed +.Pp .Sh USING SSL/TLS WITH SENDMAIL By default, .Xr sendmail 8
ls -l behavior on directories without execute permission
Hi, When ls -l is run on a directory which has no execute permissions, ls fails but the return value is 0. bash-4.2$ ls -ld /tmp/foo/ drw-r-xr-x 3 sac wheel 512 Mar 6 18:11 /tmp/foo/ bash-4.2$ ls -l /tmp/foo/ bash-4.2$ echo $? 0 bash-4.2$ I see in the traverse function: chp = fts_children(ftsp, ch_options); saved_errno = errno; display(p, chp); /* * On fts_children() returning error do recurse to see * the error. */ if (!f_recursive !(chp == NULL saved_errno != 0)) (void)fts_set(ftsp, p, FTS_SKIP); break; On the failure of fts_children function shouldn't we be setting rval = 1 and break? Since we do not anyway want to continue on a directory without execute permission as we do not get the stat information to long list the contents? Let me know if the understanding is right, I will send a patch. -sac
Remove explicit if_down() for removed trunk ports
Hi, In if_trunk.c there's an explicit if_down() for trunk ports being removed from the trunk. This seems unnecessary to me and is there since the trunk feature has initially added to OpenBSD. Greetings, Matthias === diff --git a/sys/net/if_trunk.c b/sys/net/if_trunk.c index 5ceb6cb..0b64dd8 100644 --- a/sys/net/if_trunk.c +++ b/sys/net/if_trunk.c @@ -410,10 +410,6 @@ trunk_port_destroy(struct trunk_port *tp) /* Remove multicast addresses from this port */ trunk_ether_cmdmulti(tp, SIOCDELMULTI); - /* Port has to be down */ - if (ifp-if_flags IFF_UP) - if_down(ifp); - ifpromisc(ifp, 0); /* Restore interface */ -- 1.7.9.5
Re: Kill IFAFREE()
On 05/03/13(Tue) 21:57, Claudio Jeker wrote:
On Tue, Mar 05, 2013 at 12:03:49PM +0100, Mike Belopuhov wrote:
On 5 March 2013 11:55, Mark Kettenis mark.kette...@xs4all.nl wrote:
Date: Tue, 5 Mar 2013 11:36:36 +0100
From: Martin Pieuchot mpieuc...@nolizard.org
The ifaddr structure contains a reference counter and two different way
to check it before freeing its memory: a macro IFAFREE(), and a function
ifafree().
Because the former calls the latter when the reference counter is null,
and then also check for the reference counter, I see no point in keeping
two ways to do the same thing.
Well, the point is probably that by doing the refcount check in the
macro you avoid a function call in most cases. It's very well
possible that this is a case of premature optimization. Almost
certainly the case unless the macro is called in a
performance-critical path. If it is called in a performance-critical
path, some benchmarking should probably be done to make sure this
doesn't impact something like packet forwarding performance in a
negative way.
to be fair, there are millions of these function calls. i highly
doubt one can measure any performance difference -- it'll all be
within error margin.
If we do IFAFREE() on a per packet basis then we do something wrong.
Glancing at the diff I see no hot pathes that would matter.
Exactly, we are unrefcounting address descriptors when detaching an
interface, removing addresses, modifying routes and obviously in some
ioctl() code paths.
So, ok?
Martin
diff --git sys/net/if.c sys/net/if.c
index 534d434..3edd0a7 100644
--- sys/net/if.c
+++ sys/net/if.c
@@ -599,7 +599,7 @@ do { \
continue;
ifa-ifa_ifp = NULL;
- IFAFREE(ifa);
+ ifafree(ifa);
}
for (ifg = TAILQ_FIRST(ifp-if_groups); ifg;
@@ -609,7 +609,7 @@ do { \
if_free_sadl(ifp);
ifnet_addrs[ifp-if_index]-ifa_ifp = NULL;
- IFAFREE(ifnet_addrs[ifp-if_index]);
+ ifafree(ifnet_addrs[ifp-if_index]);
ifnet_addrs[ifp-if_index] = NULL;
free(ifp-if_addrhooks, M_TEMP);
@@ -1007,7 +1007,7 @@ link_rtrequest(int cmd, struct rtentry *rt, struct
rt_addrinfo *info)
return;
if ((ifa = ifaof_ifpforaddr(dst, ifp)) != NULL) {
ifa-ifa_refcnt++;
- IFAFREE(rt-rt_ifa);
+ ifafree(rt-rt_ifa);
rt-rt_ifa = ifa;
if (ifa-ifa_rtrequest ifa-ifa_rtrequest != link_rtrequest)
ifa-ifa_rtrequest(cmd, rt, info);
@@ -1515,7 +1515,7 @@ ifioctl(struct socket *so, u_long cmd, caddr_t data,
struct proc *p)
(struct in_ifaddr *)ifa, ia_list);
ifa_del(ifp, ifa);
ifa-ifa_ifp = NULL;
- IFAFREE(ifa);
+ ifafree(ifa);
}
#endif
splx(s);
diff --git sys/net/if.h sys/net/if.h
index 26ea6b1..27b209b 100644
--- sys/net/if.h
+++ sys/net/if.h
@@ -704,14 +704,6 @@ struct if_laddrreq {
#include net/if_arp.h
#ifdef _KERNEL
-#defineIFAFREE(ifa) \
-do { \
- if ((ifa)-ifa_refcnt = 0) \
- ifafree(ifa); \
- else \
- (ifa)-ifa_refcnt--; \
-} while (/* CONSTCOND */0)
-
#ifdef ALTQ
#defineIFQ_ENQUEUE(ifq, m, pattr, err)
\
diff --git sys/net/route.c sys/net/route.c
index 9ec8a47..a0dc710 100644
--- sys/net/route.c
+++ sys/net/route.c
@@ -401,7 +401,7 @@ rtfree(struct rtentry *rt)
rt_timer_remove_all(rt);
ifa = rt-rt_ifa;
if (ifa)
- IFAFREE(ifa);
+ ifafree(ifa);
rtlabel_unref(rt-rt_labelid);
#ifdef MPLS
if (rt-rt_flags RTF_MPLS)
@@ -926,7 +926,7 @@ rtrequest1(int req, struct rt_addrinfo *info, u_int8_t prio,
if ((*ret_nrt)-rt_ifa-ifa_rtrequest)
(*ret_nrt)-rt_ifa-ifa_rtrequest(
RTM_DELETE, *ret_nrt, NULL);
- IFAFREE((*ret_nrt)-rt_ifa);
+ ifafree((*ret_nrt)-rt_ifa);
(*ret_nrt)-rt_ifa = ifa;
(*ret_nrt)-rt_ifp = ifa-ifa_ifp;
ifa-ifa_refcnt++;
@@ -957,7 +957,7 @@ rtrequest1(int req, struct rt_addrinfo *info, u_int8_t prio,
RTFREE(crt);
}
if (rn == 0) {
- IFAFREE(ifa);
+ ifafree(ifa);
if ((rt-rt_flags RTF_CLONED) != 0 rt-rt_parent)
rtfree(rt-rt_parent);
if (rt-rt_gwroute)
@@ -1139,7 +1139,7 @@
Re: ssl(8) cert generation instructions
On 2013/03/06 13:05, Stuart Henderson wrote: It's not entirely obvious that -x509 actually means produce a csr, self-sign it (defaulting to SHA1), throw away the csr and write the cert and this had me stuck for a long time when I wanted to play with DSA server certs. So here's a diff which moves DSA cert generation instructions to the same style as RSA where the process is to produce a CSR and tell people how to sign it in separate steps. It doesn't take much longer and is clearer. As a bonus there are instructions for ECDSA cert generation. OK? Tweaks from jmc and fix ..you can generate a self-signed certificate.. in the added text for ECDSA which should have been CSR. Index: ssl.8 === RCS file: /cvs/src/share/man/man8/ssl.8,v retrieving revision 1.53 diff -u -p -r1.53 ssl.8 --- ssl.8 27 Nov 2012 01:02:07 - 1.53 +++ ssl.8 6 Mar 2013 15:36:04 - @@ -150,17 +150,63 @@ The following command will generate 1024 # openssl dsaparam 1024 -out dsa1024.pem .Ed .Pp -Once you have the DSA parameters generated, you can generate a certificate -and unencrypted private key using the command: +Once you have the DSA parameters generated, you can generate a +CSR and unencrypted private key using the command: .Bd -literal -offset indent -# openssl req -x509 -nodes -newkey dsa:dsa1024.pem \e - -out /etc/ssl/dsacert.pem -keyout /etc/ssl/private/dsakey.pem +# openssl req -nodes -newkey dsa:dsa1024.pem \e + -out /etc/ssl/dsacert.csr -keyout /etc/ssl/private/dsakey.pem .Ed .Pp To generate an encrypted private key, you would use: .Bd -literal -offset indent -# openssl req -x509 -newkey dsa:dsa1024.pem \e - -out /etc/ssl/dsacert.pem -keyout /etc/ssl/private/dsakey.pem +# openssl req -newkey dsa:dsa1024.pem \e + -out /etc/ssl/dsacert.csr -keyout /etc/ssl/private/dsakey.pem +.Ed +.Pp +This +.Pa server.csr +file can then be given to a CA who will sign the key. +.Pp +You can also sign the key yourself, using the command: +.Bd -literal -offset indent +# openssl x509 -sha256 -req -days 365 \e + -in /etc/ssl/private/dsacert.csr \e + -signkey /etc/ssl/private/dsacert.key \e + -out /etc/ssl/dsacert.crt +.Ed +.Sh GENERATING ECDSA SERVER CERTIFICATES +First, generate parameters for ECDSA keys. +The following command will use a NIST/SECG curve over a 384-bit +prime field: +.Bd -literal -offset indent +# openssl ecparam -out ec-secp384r1.pem -name secp384r1 +.Ed +.Pp +Once you have the ECDSA parameters generated, you can generate a +CSR and unencrypted private key using the command: +.Bd -literal -offset indent +# openssl req -nodes -newkey ec:ec-secp384r1.pem \e + -keyout /etc/ssl/private/eccert.key -new \e + -out /etc/ssl/private/eccert.csr +.Ed +.Pp +To generate an encrypted private key, you would use: +.Bd -literal -offset indent +# openssl req -newkey ec:ec-secp384r1.pem \e + -keyout /etc/ssl/private/eccert.key -new \e + -out /etc/ssl/private/eccert.csr +.Ed +.Pp +This +.Pa eccert.csr +file can then be given to a CA who will sign the key. +.Pp +You can also sign the key yourself, using the command: +.Bd -literal -offset indent +# openssl x509 -sha256 -req -days 365 \e + -in /etc/ssl/private/eccert.csr \e + -signkey /etc/ssl/private/eccert.key \e + -out /etc/ssl/eccert.crt .Ed .Sh USING SSL/TLS WITH SENDMAIL By default,
Re: Remove explicit if_down() for removed trunk ports
On 2013/03/06 15:24, Silamael wrote: Hi, In if_trunk.c there's an explicit if_down() for trunk ports being removed from the trunk. This seems unnecessary to me and is there since the trunk feature has initially added to OpenBSD. I looked at this some time ago (because it's super annoying if you do this on a remote system to swap the active port around and forget about the behaviour ;) but I have some recollection there might have been a bit more to it than this.. anyone remember? Greetings, Matthias === diff --git a/sys/net/if_trunk.c b/sys/net/if_trunk.c index 5ceb6cb..0b64dd8 100644 --- a/sys/net/if_trunk.c +++ b/sys/net/if_trunk.c @@ -410,10 +410,6 @@ trunk_port_destroy(struct trunk_port *tp) /* Remove multicast addresses from this port */ trunk_ether_cmdmulti(tp, SIOCDELMULTI); - /* Port has to be down */ - if (ifp-if_flags IFF_UP) - if_down(ifp); - ifpromisc(ifp, 0); /* Restore interface */ -- 1.7.9.5
Re: Kill IFAFREE()
On Wed, Mar 06, 2013 at 03:58:22PM +0100, Mark Kettenis wrote:
Date: Wed, 6 Mar 2013 15:25:34 +0100
From: Martin Pieuchot mpieuc...@nolizard.org
On 05/03/13(Tue) 21:57, Claudio Jeker wrote:
On Tue, Mar 05, 2013 at 12:03:49PM +0100, Mike Belopuhov wrote:
On 5 March 2013 11:55, Mark Kettenis mark.kette...@xs4all.nl wrote:
Date: Tue, 5 Mar 2013 11:36:36 +0100
From: Martin Pieuchot mpieuc...@nolizard.org
The ifaddr structure contains a reference counter and two different
way
to check it before freeing its memory: a macro IFAFREE(), and a
function
ifafree().
Because the former calls the latter when the reference counter is
null,
and then also check for the reference counter, I see no point in
keeping
two ways to do the same thing.
Well, the point is probably that by doing the refcount check in the
macro you avoid a function call in most cases. It's very well
possible that this is a case of premature optimization. Almost
certainly the case unless the macro is called in a
performance-critical path. If it is called in a performance-critical
path, some benchmarking should probably be done to make sure this
doesn't impact something like packet forwarding performance in a
negative way.
to be fair, there are millions of these function calls. i highly
doubt one can measure any performance difference -- it'll all be
within error margin.
If we do IFAFREE() on a per packet basis then we do something wrong.
Glancing at the diff I see no hot pathes that would matter.
Exactly, we are unrefcounting address descriptors when detaching an
interface, removing addresses, modifying routes and obviously in some
ioctl() code paths.
So, ok?
I'm satisfied with the explanation. Premature optimisation is the
verdict and we all know that that's evil ;).
so ok kettenis@ (but you probably should get an ok from a real
networking person as well).
ok krw@ with same condition. :-)
Ken
diff --git sys/net/if.c sys/net/if.c
index 534d434..3edd0a7 100644
--- sys/net/if.c
+++ sys/net/if.c
@@ -599,7 +599,7 @@ do { \
continue;
ifa-ifa_ifp = NULL;
- IFAFREE(ifa);
+ ifafree(ifa);
}
for (ifg = TAILQ_FIRST(ifp-if_groups); ifg;
@@ -609,7 +609,7 @@ do { \
if_free_sadl(ifp);
ifnet_addrs[ifp-if_index]-ifa_ifp = NULL;
- IFAFREE(ifnet_addrs[ifp-if_index]);
+ ifafree(ifnet_addrs[ifp-if_index]);
ifnet_addrs[ifp-if_index] = NULL;
free(ifp-if_addrhooks, M_TEMP);
@@ -1007,7 +1007,7 @@ link_rtrequest(int cmd, struct rtentry *rt, struct
rt_addrinfo *info)
return;
if ((ifa = ifaof_ifpforaddr(dst, ifp)) != NULL) {
ifa-ifa_refcnt++;
- IFAFREE(rt-rt_ifa);
+ ifafree(rt-rt_ifa);
rt-rt_ifa = ifa;
if (ifa-ifa_rtrequest ifa-ifa_rtrequest != link_rtrequest)
ifa-ifa_rtrequest(cmd, rt, info);
@@ -1515,7 +1515,7 @@ ifioctl(struct socket *so, u_long cmd, caddr_t data,
struct proc *p)
(struct in_ifaddr *)ifa, ia_list);
ifa_del(ifp, ifa);
ifa-ifa_ifp = NULL;
- IFAFREE(ifa);
+ ifafree(ifa);
}
#endif
splx(s);
diff --git sys/net/if.h sys/net/if.h
index 26ea6b1..27b209b 100644
--- sys/net/if.h
+++ sys/net/if.h
@@ -704,14 +704,6 @@ struct if_laddrreq {
#include net/if_arp.h
#ifdef _KERNEL
-#defineIFAFREE(ifa) \
-do { \
- if ((ifa)-ifa_refcnt = 0) \
- ifafree(ifa); \
- else \
- (ifa)-ifa_refcnt--; \
-} while (/* CONSTCOND */0)
-
#ifdef ALTQ
#defineIFQ_ENQUEUE(ifq, m, pattr, err)
\
diff --git sys/net/route.c sys/net/route.c
index 9ec8a47..a0dc710 100644
--- sys/net/route.c
+++ sys/net/route.c
@@ -401,7 +401,7 @@ rtfree(struct rtentry *rt)
rt_timer_remove_all(rt);
ifa = rt-rt_ifa;
if (ifa)
- IFAFREE(ifa);
+ ifafree(ifa);
rtlabel_unref(rt-rt_labelid);
#ifdef MPLS
if (rt-rt_flags RTF_MPLS)
@@ -926,7 +926,7 @@ rtrequest1(int req, struct rt_addrinfo *info, u_int8_t
prio,
if ((*ret_nrt)-rt_ifa-ifa_rtrequest)
(*ret_nrt)-rt_ifa-ifa_rtrequest(
RTM_DELETE, *ret_nrt, NULL);
- IFAFREE((*ret_nrt)-rt_ifa);
+ ifafree((*ret_nrt)-rt_ifa);
(*ret_nrt)-rt_ifa = ifa;
(*ret_nrt)-rt_ifp = ifa-ifa_ifp;
Re: ssl(8) cert generation instructions
On Wed, Mar 06, 2013 at 01:05:16PM +, Stuart Henderson wrote: It's not entirely obvious that -x509 actually means produce a csr, self-sign it (defaulting to SHA1), throw away the csr and write the cert and this had me stuck for a long time when I wanted to play with DSA server certs. So here's a diff which moves DSA cert generation instructions to the same style as RSA where the process is to produce a CSR and tell people how to sign it in separate steps. It doesn't take much longer and is clearer. As a bonus there are instructions for ECDSA cert generation. OK? I'd like to mention in passing that I got bitten recently by the default lifetime limit of just 30 days for certs. I created my own CA but could only use it for one month :( Perhaps that could be mentioned. Or a -days option could be added to the example.
spamlogd whitelists every logged rdr-to connection
Hi,
I've started using spamlogd, and since then, every single
connection attempt results in the host being whitelisted.
I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog,
and it would seem like spamlogd filter (for port 25)
is picking up the original dport, not the rewritten one
(with hdr-dport containing original port, too).
Not sure of the correct solution, but one of the options is
to look at the hdr-rewritten field, and only act if it is false.
This might impact someone who does pf rewrites for sendmail itself,
but at least it's not going to let all the spam in for someone
who simply logs stuff up. A patch is attached.
Cheers,
Constantine.
Cns# tail /var/log/spamd
Mar 6 08:12:53 Cns spamlogd[1082]: inbound 74.122.155.17
Mar 6 08:50:27 Cns spamd[5220]: 46.53.132.165: connected (1/0)
Mar 6 08:50:27 Cns spamlogd[1082]: inbound 46.53.132.165
Mar 6 08:50:30 Cns spamd[5220]: 46.53.132.165: disconnected after 3 seconds.
Mar 6 08:51:37 Cns spamd[5220]: 178.127.228.161: connected (1/0)
Mar 6 08:51:37 Cns spamlogd[1082]: inbound 178.127.228.161
Mar 6 08:51:40 Cns spamd[5220]: 178.127.228.161: disconnected after 3 seconds.
Mar 6 09:21:54 Cns spamlogd[1082]: inbound 46.241.252.81
Mar 6 09:21:55 Cns spamd[5220]: 46.241.252.81: connected (1/0)
Mar 6 09:21:58 Cns spamd[5220]: 46.241.252.81: disconnected after 3 seconds.
Cns# fgrep 46.241.252.81 /var/log/spamd
Mar 6 09:21:54 Cns spamlogd[1082]: inbound 46.241.252.81
Mar 6 09:21:55 Cns spamd[5220]: 46.241.252.81: connected (1/0)
Mar 6 09:21:58 Cns spamd[5220]: 46.241.252.81: disconnected after 3 seconds.
Cns# tcpdump -o -n -e -ttt -r /var/log/pflog host 46.241.252.81 | tail
tcpdump: WARNING: snaplen raised from 116 to 160
Mar 06 09:21:54.834606 rule 43/(match) pass in on re0: 46.241.252.81.3748
127.0.0.1.8025: S (src OS: Windows 2000 RFC1323, Windows XP RFC1323)
4277363076:4277363076(0) win 65535 mss 1460,nop,wscale 2,nop,nop,sackOK (DF)
Cns# fgrep 46.241.252.81 /var/log/maillog
Cns# spamdb | fgrep 46.241.252.81
WHITE|46.241.252.81|||1362590514|1362590514|1365700914|1|0
Cns# date -r 1362590514
Wed Mar 6 09:21:54 PST 2013
Cns# uname -rms
OpenBSD 5.2 amd64
(Logs were rorated several days ago, so, what you see is what's there.)
Index: spamlogd.c
===
RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
retrieving revision 1.21
diff -u -d -p -8 -r1.21 spamlogd.c
--- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
+++ spamlogd.c 6 Mar 2013 19:44:32 -
@@ -174,20 +174,22 @@ logpkt_handler(u_char *user, const struc
/* We're interested in passed packets */
if (hdr-action != PF_PASS)
return;
af = hdr-af;
if (af == AF_INET) {
ip = (const struct ip *)(sp + hdrlen);
- if (hdr-dir == PF_IN)
+ if (hdr-dir == PF_IN) {
+ if (hdr-rewritten == 1)
+ return;
inet_ntop(af, ip-ip_src, ipstraddr,
sizeof(ipstraddr));
- else if (hdr-dir == PF_OUT !flag_inbound)
+ } else if (hdr-dir == PF_OUT !flag_inbound)
inet_ntop(af, ip-ip_dst, ipstraddr,
sizeof(ipstraddr));
}
if (ipstraddr[0] != '\0') {
if (hdr-dir == PF_IN)
logmsg(LOG_DEBUG,inbound %s, ipstraddr);
else
Re: Fuse (and sshfs) support for OpenBSD
On Wed, Mar 06, 2013 at 22:27, Sylvestre Gallon wrote: Do you know if miscfs is the best place to put my code ? I think so. Do I keep the device code (fuse_device.c) in the same directory than the filesystem code? I'd prefer that. sys/dev is kind of cluttered as it is, and since the dev code and fs code are tightly coupled, it makes sense to keep it all in one place. Do you know which fs type I can use in kern/vfs_init.c for fuse ? I've taken 42 but I have no idea what I was doing when I choose this number :) Whatever. I don't think those numbers are used. +/* + * The root inode is the root of the file system. Inode 0 can't be used for + * normal purposes and bad blocks are normally linked to inode 1, thus + * the root inode is 2. + */ +#defineFUSE_ROOTINO ((ino_t)1) Internal disagreement here. :)
Re: spamlogd whitelists every logged rdr-to connection
Bob, I agree, the hdr-rewritten approach is not good.
I think the best approach here would be to not add any new entries
on incoming connections in the first place, but only keep updating
the existing ones (when the connection is incoming).
In addition to not whitelisting greylisted or blocked connection that
are logged, this would also prevent the case of double-whitelisting
the connections that are logged and whitelisted through other rules,
without any adverse side effects or unexpected behaviour.
Patch attached inline.
C.
On 2013-W10-3 13:47 -0700, Bob Beck wrote:
No constantine - the solution is to simply not use the log keyword
on such traffic
All of my boxen I run this on also rewite the traffic to (pool) of
mailservers so this is
not accurate.
Simply don't log the traffic you don't want spamlogd to see. the
*point* of spamlogd
is to ensure all continuing valid connections *stay* whitelisted.
On Wed, Mar 6, 2013 at 1:08 PM, Constantine A. Murenin c...@cns.su wrote:
Hi,
I've started using spamlogd, and since then, every single connection attempt
results in the host being whitelisted.
I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog, and
it would seem like spamlogd filter (for port 25) is picking up the original
dport, not the rewritten one (with hdr-dport containing original port,
too).
Not sure of the correct solution, but one of the options is to look at the
hdr-rewritten field, and only act if it is false. This might impact
someone who does pf rewrites for sendmail itself, but at least it's not
going to let all the spam in for someone who simply logs stuff up. A patch
is attached.
Cheers,
Constantine.
Index: spamlogd.c
===
RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
retrieving revision 1.21
diff -u -d -p -8 -r1.21 spamlogd.c
--- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
+++ spamlogd.c 6 Mar 2013 21:14:51 -
@@ -75,17 +75,17 @@ pcap_t *hpcap = NULL;
struct syslog_data sdata = SYSLOG_DATA_INIT;
time_t whiteexp = WHITEEXP;
extern char*__progname;
void logmsg(int , const char *, ...);
void sighandler_close(int);
intinit_pcap(void);
void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char *);
-intdbupdate(char *, char *);
+intdbupdate(char *, char *, int);
void usage(void);
void
logmsg(int pri, const char *msg, ...)
{
va_list ap;
va_start(ap, msg);
@@ -187,22 +187,22 @@ logpkt_handler(u_char *user, const struc
sizeof(ipstraddr));
}
if (ipstraddr[0] != '\0') {
if (hdr-dir == PF_IN)
logmsg(LOG_DEBUG,inbound %s, ipstraddr);
else
logmsg(LOG_DEBUG,outbound %s, ipstraddr);
- dbupdate(PATH_SPAMD_DB, ipstraddr);
+ dbupdate(PATH_SPAMD_DB, ipstraddr, hdr-dir == PF_IN);
}
}
int
-dbupdate(char *dbname, char *ip)
+dbupdate(char *dbname, char *ip, int updateonly)
{
HASHINFOhashinfo;
DBT dbk, dbd;
DB *db;
struct gdatagd;
time_t now;
int r;
struct in_addr ia;
@@ -227,16 +227,20 @@ dbupdate(char *dbname, char *ip)
/* add or update whitelist entry */
r = db-get(db, dbk, dbd, 0);
if (r == -1) {
logmsg(LOG_NOTICE, db-get failed (%m));
goto bad;
}
if (r) {
+ if (updateonly) {
+ logmsg(LOG_DEBUG,ignoring %s, ip);
+ goto bad;
+ }
/* new entry */
memset(gd, 0, sizeof(gd));
gd.first = now;
gd.bcount = 1;
gd.pass = now;
gd.expire = now + whiteexp;
memset(dbk, 0, sizeof(dbk));
dbk.size = strlen(ip);
Re: spamlogd whitelists every logged rdr-to connection
No constatine - that is not the best approach. if you are whitelisting
grelisted connections
or blacklisted connections that are blocked you have your pf.conf or
spamlogd setup wrong.
On Wed, Mar 6, 2013 at 2:54 PM, Constantine A. Murenin c...@cns.su wrote:
Bob, I agree, the hdr-rewritten approach is not good.
I think the best approach here would be to not add any new entries on
incoming connections in the first place, but only keep updating the existing
ones (when the connection is incoming).
In addition to not whitelisting greylisted or blocked connection that are
logged, this would also prevent the case of double-whitelisting the
connections that are logged and whitelisted through other rules, without any
adverse side effects or unexpected behaviour.
Patch attached inline.
C.
On 2013-W10-3 13:47 -0700, Bob Beck wrote:
No constantine - the solution is to simply not use the log keyword
on such traffic
All of my boxen I run this on also rewite the traffic to (pool) of
mailservers so this is
not accurate.
Simply don't log the traffic you don't want spamlogd to see. the
*point* of spamlogd
is to ensure all continuing valid connections *stay* whitelisted.
On Wed, Mar 6, 2013 at 1:08 PM, Constantine A. Murenin c...@cns.su wrote:
Hi,
I've started using spamlogd, and since then, every single connection
attempt
results in the host being whitelisted.
I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog,
and
it would seem like spamlogd filter (for port 25) is picking up the
original
dport, not the rewritten one (with hdr-dport containing original port,
too).
Not sure of the correct solution, but one of the options is to look at
the
hdr-rewritten field, and only act if it is false. This might impact
someone who does pf rewrites for sendmail itself, but at least it's not
going to let all the spam in for someone who simply logs stuff up. A
patch
is attached.
Cheers,
Constantine.
Index: spamlogd.c
===
RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
retrieving revision 1.21
diff -u -d -p -8 -r1.21 spamlogd.c
--- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
+++ spamlogd.c 6 Mar 2013 21:14:51 -
@@ -75,17 +75,17 @@ pcap_t *hpcap = NULL;
struct syslog_data sdata = SYSLOG_DATA_INIT;
time_t whiteexp = WHITEEXP;
extern char*__progname;
void logmsg(int , const char *, ...);
void sighandler_close(int);
intinit_pcap(void);
void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char
*);
-intdbupdate(char *, char *);
+intdbupdate(char *, char *, int);
void usage(void);
void
logmsg(int pri, const char *msg, ...)
{
va_list ap;
va_start(ap, msg);
@@ -187,22 +187,22 @@ logpkt_handler(u_char *user, const struc
sizeof(ipstraddr));
}
if (ipstraddr[0] != '\0') {
if (hdr-dir == PF_IN)
logmsg(LOG_DEBUG,inbound %s, ipstraddr);
else
logmsg(LOG_DEBUG,outbound %s, ipstraddr);
- dbupdate(PATH_SPAMD_DB, ipstraddr);
+ dbupdate(PATH_SPAMD_DB, ipstraddr, hdr-dir == PF_IN);
}
}
int
-dbupdate(char *dbname, char *ip)
+dbupdate(char *dbname, char *ip, int updateonly)
{
HASHINFOhashinfo;
DBT dbk, dbd;
DB *db;
struct gdatagd;
time_t now;
int r;
struct in_addr ia;
@@ -227,16 +227,20 @@ dbupdate(char *dbname, char *ip)
/* add or update whitelist entry */
r = db-get(db, dbk, dbd, 0);
if (r == -1) {
logmsg(LOG_NOTICE, db-get failed (%m));
goto bad;
}
if (r) {
+ if (updateonly) {
+ logmsg(LOG_DEBUG,ignoring %s, ip);
+ goto bad;
+ }
/* new entry */
memset(gd, 0, sizeof(gd));
gd.first = now;
gd.bcount = 1;
gd.pass = now;
gd.expire = now + whiteexp;
memset(dbk, 0, sizeof(dbk));
dbk.size = strlen(ip);
Re: spamlogd whitelists every logged rdr-to connection
Indeed, this is what I typically do. and make sure I only log the real mailserver connections to pflog1, and point spamlogd at that. On Wed, Mar 6, 2013 at 2:14 PM, Stuart Henderson s...@spacehopper.org wrote: On 2013/03/06 13:47, Bob Beck wrote: No constantine - the solution is to simply not use the log keyword on such traffic Or you can use an alternative log interface log (to interface) Send logs to the specified pflog(4) interface instead of pflog0.
Re: spamlogd whitelists every logged rdr-to connection
I'm simply logging greylisted connections;
it's spamlogd that whitelists them just because they're logged.
It doesn't make sense that logging greylisted or blacklisted connections
would immediately turn them into being whitelisted by spamlogd.
Same goes for logging connections that are already whitelisted
through rules and tables other than spamd-white.
Why would you want them whitelisted several times?
My second patch fixes these issues, without breaking valid setups.
Yes, one could log stuff into different pflog interfaces, but
I don't understand why pf.conf `pass in ... log ... port smtp ...`
is effectively redefined to mean `add spamd-white`
when spamlogd is running, even when connections are redirected
to spamd for stuttering or greylisting.
That's not something that seems reasonable, and an update-if-exists
logic would make so much more sense.
C.
On 2013-W10-3 14:56 -0700, Bob Beck wrote:
No constatine - that is not the best approach. if you are whitelisting
grelisted connections
or blacklisted connections that are blocked you have your pf.conf or
spamlogd setup wrong.
On Wed, Mar 6, 2013 at 2:54 PM, Constantine A. Murenin c...@cns.su wrote:
Bob, I agree, the hdr-rewritten approach is not good.
I think the best approach here would be to not add any new entries on
incoming connections in the first place, but only keep updating the existing
ones (when the connection is incoming).
In addition to not whitelisting greylisted or blocked connection that are
logged, this would also prevent the case of double-whitelisting the
connections that are logged and whitelisted through other rules, without any
adverse side effects or unexpected behaviour.
Patch attached inline.
C.
On 2013-W10-3 13:47 -0700, Bob Beck wrote:
No constantine - the solution is to simply not use the log keyword
on such traffic
All of my boxen I run this on also rewite the traffic to (pool) of
mailservers so this is
not accurate.
Simply don't log the traffic you don't want spamlogd to see. the
*point* of spamlogd
is to ensure all continuing valid connections *stay* whitelisted.
On Wed, Mar 6, 2013 at 1:08 PM, Constantine A. Murenin c...@cns.su wrote:
Hi,
I've started using spamlogd, and since then, every single connection
attempt
results in the host being whitelisted.
I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog,
and
it would seem like spamlogd filter (for port 25) is picking up the
original
dport, not the rewritten one (with hdr-dport containing original port,
too).
Not sure of the correct solution, but one of the options is to look at
the
hdr-rewritten field, and only act if it is false. This might impact
someone who does pf rewrites for sendmail itself, but at least it's not
going to let all the spam in for someone who simply logs stuff up. A
patch
is attached.
Cheers,
Constantine.
Index: spamlogd.c
===
RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
retrieving revision 1.21
diff -u -d -p -8 -r1.21 spamlogd.c
--- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
+++ spamlogd.c 6 Mar 2013 21:14:51 -
@@ -75,17 +75,17 @@ pcap_t *hpcap = NULL;
struct syslog_data sdata = SYSLOG_DATA_INIT;
time_t whiteexp = WHITEEXP;
extern char*__progname;
void logmsg(int , const char *, ...);
void sighandler_close(int);
intinit_pcap(void);
void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char
*);
-intdbupdate(char *, char *);
+intdbupdate(char *, char *, int);
void usage(void);
void
logmsg(int pri, const char *msg, ...)
{
va_list ap;
va_start(ap, msg);
@@ -187,22 +187,22 @@ logpkt_handler(u_char *user, const struc
sizeof(ipstraddr));
}
if (ipstraddr[0] != '\0') {
if (hdr-dir == PF_IN)
logmsg(LOG_DEBUG,inbound %s, ipstraddr);
else
logmsg(LOG_DEBUG,outbound %s, ipstraddr);
- dbupdate(PATH_SPAMD_DB, ipstraddr);
+ dbupdate(PATH_SPAMD_DB, ipstraddr, hdr-dir == PF_IN);
}
}
int
-dbupdate(char *dbname, char *ip)
+dbupdate(char *dbname, char *ip, int updateonly)
{
HASHINFOhashinfo;
DBT dbk, dbd;
DB *db;
struct gdatagd;
time_t now;
int r;
struct in_addr ia;
@@ -227,16 +227,20 @@ dbupdate(char *dbname, char *ip)
/* add or update whitelist entry */
r = db-get(db, dbk, dbd, 0);
if (r == -1) {
logmsg(LOG_NOTICE, db-get failed (%m));
goto bad;
}
if (r) {
+ if (updateonly) {
+ logmsg(LOG_DEBUG,ignoring %s,
Re: spamlogd whitelists every logged rdr-to connection
Yes, one could log stuff into different pflog interfaces, but I don't understand why pf.conf `pass in ... log ... port smtp ...` is effectively redefined to mean `add spamd-white` when spamlogd is running, http://www.openbsd.org/cgi-bin/man.cgi?query=spamlogd and RTFM for the first two sentences - and it's pretty darn clear. Those of us that whitelist blocks of addresses (and log them) like this behaviour to track what mailservers we are seeing like the current behavior.
