Re: mmap diff for patch(1)
On Fri, Nov 11, 2011 at 03:04:28PM +0800, Michael W. Bombardieri wrote:
Sorry, let me try that again... Forgot to clean up file descriptor ifd.
AFAIK, this works without this diff. What problem are you trying to solve?
-Otto
Index: inp.c
===
RCS file: /usr/src/cvs/src/usr.bin/patch/inp.c,v
retrieving revision 1.35
diff -u -r1.35 inp.c
--- inp.c 27 Oct 2009 23:59:41 - 1.35
+++ inp.c 11 Nov 2011 06:51:13 -
@@ -243,6 +243,10 @@
if ((ifd = open(filename, O_RDONLY)) 0)
pfatal(can't open file %s, filename);
+ if (i_size == 0) {
+ close(ifd);
+ return true;
+ }
i_womp = mmap(NULL, i_size, PROT_READ, MAP_PRIVATE, ifd, 0);
if (i_womp == MAP_FAILED) {
perror(mmap failed);
Re: mmap diff for patch(1)
On Fri, Nov 11, 2011 at 10:20:19AM +0100, Otto Moerbeek wrote: On Fri, Nov 11, 2011 at 03:04:28PM +0800, Michael W. Bombardieri wrote: Sorry, let me try that again... Forgot to clean up file descriptor ifd. AFAIK, this works without this diff. What problem are you trying to solve? Some systems reject mmap with size 0. That's what the original chance was all about. Joerg
smtpd: fatal: qwalk: opendir: No such file or directory
I have no clue what is wrong here. maybe just the '0' before 'ab3' got lost? % sudo smtpd -vd startup [debug mode] parent_send_config: configuring smtp parent_send_config_client_certs: configuring smtp parent_send_config_ruleset: reloading rules and maps parent_send_config_ruleset: reloading rules and maps ramqueue: queue loading in progress q-path is /queue/ab3 -- fprintf line added by me. fatal: qwalk: opendir: No such file or directory queue handler exiting control process exiting lost child: runner exited abnormally mail filter exiting mail delivery agent exiting smtp server exiting lookup agent exiting mail transfer agent exiting parent terminating % sudo ls -l /var/spool/smtpd/queue total 16 drwx-- 3 _smtpd wheel 512 May 8 2011 0151 drwx-- 3 _smtpd wheel 512 Jul 14 15:12 0425 drwx-- 3 _smtpd wheel 512 May 8 2011 0ab3 drwx-- 3 _smtpd wheel 512 May 8 2011 0f25
Re: smtpd: fatal: qwalk: opendir: No such file or directory
Mh, Did you update OpenSMTPD while your queue was not empty ? You should just rename the directories by removing the prepended 0, it should fix your issue. Gilles On Fri, Nov 11, 2011 at 04:30:52PM +0100, Christopher Zimmermann wrote: I have no clue what is wrong here. maybe just the '0' before 'ab3' got lost? % sudo smtpd -vd startup [debug mode] parent_send_config: configuring smtp parent_send_config_client_certs: configuring smtp parent_send_config_ruleset: reloading rules and maps parent_send_config_ruleset: reloading rules and maps ramqueue: queue loading in progress q-path is /queue/ab3 -- fprintf line added by me. fatal: qwalk: opendir: No such file or directory queue handler exiting control process exiting lost child: runner exited abnormally mail filter exiting mail delivery agent exiting smtp server exiting lookup agent exiting mail transfer agent exiting parent terminating % sudo ls -l /var/spool/smtpd/queue total 16 drwx-- 3 _smtpd wheel 512 May 8 2011 0151 drwx-- 3 _smtpd wheel 512 Jul 14 15:12 0425 drwx-- 3 _smtpd wheel 512 May 8 2011 0ab3 drwx-- 3 _smtpd wheel 512 May 8 2011 0f25 -- Gilles Chehade http://www.poolp.org@poolpOrg
Re: smtpd: fatal: qwalk: opendir: No such file or directory
On an unrelated note, I am almost done with a qwalk rewrite that makes it more tolerant to errors amongst other things Gilles On Fri, Nov 11, 2011 at 04:40:26PM +0100, Gilles Chehade wrote: Mh, Did you update OpenSMTPD while your queue was not empty ? You should just rename the directories by removing the prepended 0, it should fix your issue. Gilles On Fri, Nov 11, 2011 at 04:30:52PM +0100, Christopher Zimmermann wrote: I have no clue what is wrong here. maybe just the '0' before 'ab3' got lost? % sudo smtpd -vd startup [debug mode] parent_send_config: configuring smtp parent_send_config_client_certs: configuring smtp parent_send_config_ruleset: reloading rules and maps parent_send_config_ruleset: reloading rules and maps ramqueue: queue loading in progress q-path is /queue/ab3 -- fprintf line added by me. fatal: qwalk: opendir: No such file or directory queue handler exiting control process exiting lost child: runner exited abnormally mail filter exiting mail delivery agent exiting smtp server exiting lookup agent exiting mail transfer agent exiting parent terminating % sudo ls -l /var/spool/smtpd/queue total 16 drwx-- 3 _smtpd wheel 512 May 8 2011 0151 drwx-- 3 _smtpd wheel 512 Jul 14 15:12 0425 drwx-- 3 _smtpd wheel 512 May 8 2011 0ab3 drwx-- 3 _smtpd wheel 512 May 8 2011 0f25 -- Gilles Chehade http://www.poolp.org @poolpOrg -- Gilles Chehade http://www.poolp.org@poolpOrg
relayd should verify config before (re)loading
relayd doesn't verify a config file before it tries to load it, unlike many
of the other daemons (for instance, bgpd). This leads to all of the relays,
etc., being dropped if you forget to manually check the file with 'relayd
-n' before issuing a reload request. It looks to be a trivial change to
make it act more like bgpd and stop processing if the config file is bad.
A diff is at the end of this mail. As trivial as it is, this is my
biggest patch submission so far, so if it's not right let me know how I
can improve it. Thanks!
Seth
Original:
-
# relayctl show summary
Id TypeNameAvlblty Status
1 relay test_exchange_http active
1 table test_hubs:80empty
1 host134.126.52.201 0.00% down
2 host134.126.52.202 0.00% down
2 relay www_jmu_edu active
2 table www:80 active (1 hosts)
3 host134.126.10.50 100.00% up
# relayctl load relayd.conf.bad
# relayctl show summary
Id TypeNameAvlblty Status
-
Original's logging bits:
-
parent_reload: level 0 config file relayd.conf.bad
relayd.conf.bad:25: macro 'local' not defined
relayd.conf.bad:25: syntax error
relayd.conf.bad:32: macro 'local' not defined
relayd.conf.bad:32: syntax error
warning: macro '1local' not used
no actions, nothing to do
unused protocol: www
unused protocol: owa
parent_reload: failed to load config file relayd.conf.bad
-
Modified:
-
# obj/relayctl show summary
Id TypeNameAvlblty Status
1 relay test_exchange_http active
1 table test_hubs:80active (2 hosts)
1 host134.126.52.201 100.00% up
2 host134.126.52.202 100.00% up
2 relay www_jmu_edu active
2 table www:80 active (1 hosts)
3 host134.126.10.50 100.00% up
# obj/relayctl load relayd.conf.bad
load request sent.
# obj/relayctl show summary
Id TypeNameAvlblty Status
1 relay test_exchange_http active
1 table test_hubs:80active (2 hosts)
1 host134.126.52.201 100.00% up
2 host134.126.52.202 100.00% up
2 relay www_jmu_edu active
2 table www:80 active (1 hosts)
3 host134.126.10.50 100.00% up
-
Modified's logging bits:
-
parent_reload: level 0 config file relayd.conf.bad
relayd.conf.bad:25: macro 'local' not defined
relayd.conf.bad:25: syntax error
relayd.conf.bad:32: macro 'local' not defined
relayd.conf.bad:32: syntax error
config file relayd.conf.bad has errors, not reloading
-
Index: relayctl/relayctl.c
===
RCS file: /cvs/src/usr.sbin/relayctl/relayctl.c,v
retrieving revision 1.45
diff -u -p -r1.45 relayctl.c
--- relayctl/relayctl.c 20 May 2011 09:43:53 - 1.45
+++ relayctl/relayctl.c 11 Nov 2011 15:30:14 -
@@ -182,10 +182,12 @@ main(int argc, char *argv[])
case LOAD:
imsg_compose(ibuf, IMSG_CTL_RELOAD, 0, 0, -1,
res-path, strlen(res-path));
+ printf(load request sent.\n);
done = 1;
break;
case RELOAD:
imsg_compose(ibuf, IMSG_CTL_RELOAD, 0, 0, -1, NULL, 0);
+ printf(reload request sent.\n);
done = 1;
break;
case MONITOR:
Index: relayd/relayd.c
===
RCS file: /cvs/src/usr.sbin/relayd/relayd.c,v
retrieving revision 1.104
diff -u -p -r1.104 relayd.c
--- relayd/relayd.c 4 Sep 2011 20:26:58 - 1.104
+++ relayd/relayd.c 10 Nov 2011 16:53:20 -
@@ -335,6 +335,12 @@ parent_reload(struct relayd *env, u_int
log_debug(%s: level %d config file %s, __func__, reset, filename);
+ if (parse_config(filename, env) == -1) {
+ log_warnx(config file %s has errors, not reloading,
+ filename);
+ return;
+ }
+
config_purge(env, CONFIG_ALL);
if (reset ==
Re: mmap diff for patch(1)
I don't see any reason to even open the file if the size is 0.
On Fri, Nov 11, 2011, Michael W. Bombardieri wrote:
Sorry, let me try that again... Forgot to clean up file descriptor ifd.
Index: inp.c
===
RCS file: /usr/src/cvs/src/usr.bin/patch/inp.c,v
retrieving revision 1.35
diff -u -r1.35 inp.c
--- inp.c 27 Oct 2009 23:59:41 - 1.35
+++ inp.c 11 Nov 2011 06:51:13 -
@@ -243,6 +243,10 @@
if ((ifd = open(filename, O_RDONLY)) 0)
pfatal(can't open file %s, filename);
+ if (i_size == 0) {
+ close(ifd);
+ return true;
+ }
i_womp = mmap(NULL, i_size, PROT_READ, MAP_PRIVATE, ifd, 0);
if (i_womp == MAP_FAILED) {
perror(mmap failed);
increase MAXPHYS?
Is there a reason MAXPHYS (currently = 64 * 1024) should not be raised, at least on i386/amd64? It appears that 64K transfers on a 120 MB/sec disk incurs about 1800 interrupts/second. The limit seems low for a modern system and the interrupt rate is high. Does anyone see bad consequences from raising MAXPHYS to 256K and reducing the interrupt rate? Yes, it would mean more pages locked down for larger transfers. I ask this because on almost identical motherboards and CPUs and exactly identical drives, Linux is faster (approx. 143MB/sec vs. 114MB/sec). Linux is using a *lot* more CPU (approx 20% system interrupt time) OpenBSD is almost idle. It is taking about 1000 interrupts per second that appear to be disk-related. My guess from the Linux vmstat numbers is that it is reading 128K or more at a time - approx 1000 transfers/sec and approx 143 MB/sec. It may be queueing readahead requests which would overlap interrupt service with hardware I/O - I don't know. uname -a OpenBSD lib.oat.com 4.9 GENERIC.MP#819 amd64 dd if=/dev/rsd1c bs=2m of=/dev/null count=1 1+0 records in 1+0 records out 2097152 bytes transferred in 184.068 secs (113932934 bytes/sec) uname -a Linux ping 2.6.32-34-generic-pae #77-Ubuntu SMP Tue Sep 13 21:16:18 UTC 2011 i686 GNU/Linux dd if=/dev/sdb of=/dev/null bs=2M count=1 1+0 records in 1+0 records out 2097152 bytes (21 GB) copied, 147.096 s, 143 MB/s It's easy to make this change. Finding any bad consequences would be harder, and I'd like any available wisdom first. I have a reason to push disks as fast as hardware allows and want to reduce software-induced bottlenecks as much as possible. thanks! Geoff Steckel
scratch increasing MAXPHYS
Increasing MAXPHYS to 256K shows a few places where it's assumed that there are 16 pages in MAXPHYS. In dev/ic/ahci.c I had to make this change @307 to make the scatter-gather table large enough - 1 entry per page + extra because that's what the previous code had and didn't say why. I could understand +1 because a lot of code works that way. /* this makes ahci_cmd_table 512 bytes, supporting 128-byte alignment */ /* #define AHCI_MAX_PRDT24 too small for 256K of 4K pages */ /* extra 12 is to match old 16 + 8 */ #define AHCI_MAX_PRDT ((MAXPHYS / PAGE_SIZE) + 8) Grep-ing shows at least dev/ic/osiopvar.h doesn't compute DMA resources from MAXPHYS. There are probably other 17s buried in ugly places. It doesn't seem to help disk I/O speed at all. It *does* decrease interrupt rate to about 400/sec. Now to try some other tests. Grumble. Geoff Steckel
Re: scratch increasing MAXPHYS
its generally hard for the cpu to slow a disk down... On 12/11/2011, at 8:36 AM, Geoff Steckel wrote: Increasing MAXPHYS to 256K shows a few places where it's assumed that there are 16 pages in MAXPHYS. In dev/ic/ahci.c I had to make this change @307 to make the scatter-gather table large enough - 1 entry per page + extra because that's what the previous code had and didn't say why. I could understand +1 because a lot of code works that way. /* this makes ahci_cmd_table 512 bytes, supporting 128-byte alignment */ /* #define AHCI_MAX_PRDT24 too small for 256K of 4K pages */ /* extra 12 is to match old 16 + 8 */ #define AHCI_MAX_PRDT ((MAXPHYS / PAGE_SIZE) + 8) Grep-ing shows at least dev/ic/osiopvar.h doesn't compute DMA resources from MAXPHYS. There are probably other 17s buried in ugly places. It doesn't seem to help disk I/O speed at all. It *does* decrease interrupt rate to about 400/sec. Now to try some other tests. Grumble. Geoff Steckel
