Re: Double buffered cp(1)
:You are right but that is because I haven't started keeping record on :4.0-Stable and we were comparing apples and oranges. A buildworld of :3.4-Stable required around 2000u seconds using gcc-2.8.2 on my system. :Setiathome, which is running at a nice of 19, still consumed 90% of :the cpu. A buildworld on 4.0-Stable required 3500u seconds using :gcc-2.95.2 and setiathome didn't accrue any appreciable cpu time :during the build. There were definitely some changes there :). : :Kent : :-- :Kent Stewart :Richland, WA Both 3.4 and 4.0 buildworlds are cpu-bound. If you are trying to test buildworlds, then don't run setiathome (or anything else) while doing the test... it will skew the results of your tests due to differences between the 3.4 and 4.x schedulers (specifically, various scheduler bugs were fixed in 4.x that effect niced cpu-bound background programs such as setiathome, giving them way, way too much cpu). It is simply impossible to fairly measure I/O performance in the presence of unrelated background-running programs, especially under 3.x. And even though 4.x does a better job of it, it will still skew the results. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Double buffered cp(1)
Matthew Dillon wrote: :You are right but that is because I haven't started keeping record on :4.0-Stable and we were comparing apples and oranges. A buildworld of :3.4-Stable required around 2000u seconds using gcc-2.8.2 on my system. :Setiathome, which is running at a nice of 19, still consumed 90% of :the cpu. A buildworld on 4.0-Stable required 3500u seconds using :gcc-2.95.2 and setiathome didn't accrue any appreciable cpu time :during the build. There were definitely some changes there :). : :Kent : :-- :Kent Stewart :Richland, WA Both 3.4 and 4.0 buildworlds are cpu-bound. If you are trying to test buildworlds, then don't run setiathome (or anything else) while doing the test... it will skew the results of your tests due to differences between the 3.4 and 4.x schedulers (specifically, various scheduler bugs were fixed in 4.x that effect niced cpu-bound background programs such as setiathome, giving them way, way too much cpu). The bugs were fixed in 4.0? What I saw was far too much cpu going to setiathome on 3.4. Seti hardly ran on 4.0. I don't have quantities because I have only noticed the really large increase in cpu time required to build 4.0. The wall clock time on a buildworld hardly changed (55-60 minutes) whether I ran seti or not on 3.4. I can watch the wall clock time on some of my future builds and see how they are skewed by stopping seti before I being the buildworld. I just haven't got 4.0 to the capability I had with with 3.4 before I tried to upgrade to 4.0 and it died in the middle of creating the installkernel. The rest of the system was pretty much broken at that point and I used the opportunity to restructure everything. It has been a good check on some of the ports because I found a few that assumed you have things like Bison, automake, and autoconf installed and I didn't. It is simply impossible to fairly measure I/O performance in the presence of unrelated background-running programs, especially under 3.x. And even though 4.x does a better job of it, it will still skew the results. I was looking at this as more of a real world setup simulation. Seti is almost pure cpu and the buildworld used everything else. I ran the build world from an x-term and from the command line. That didn't seem to matter much. The system also provided my dialup and nat'ing for my internal network. Seti was run from a script that was started from an altF2 login before I did my startx. It would chug along from one network problem at Berkeley until the next one with out any intervention on my part. The system was on a UPS and didn't go down on the occasional 1 or 2 second long power outages. Between the two codes, they mimic most of the types of calculations I've been associated with for many years. I have people that are using Windows 9x machines and I think they would be better off with something like FreeBSD. The programs were developed in unix environment. A lot of users are using Linux. Some are using PVM to combine systems into a single parallel virtual calculation. Lehey Fortran-90(77) running on Win9x with their protected mode interface setup has to be a terrible choice. The problem is proving it and providing an alternative :). A couple would run better on a scsi because of the queued read/write I/O that you identified. You can say anything is a POS but people won't listen unless you can show them a better way. I'm retired and no longer have a contract manager to answer to and can experiment. Cheers, Kent -Matt -- Kent Stewart Richland, WA mailto:[EMAIL PROTECTED] http://www.3-cities.com/~kstewart/index.html FreeBSD News http://daily.daemonnews.org/ SETI(Search for Extraterrestrial Intelligence) @ HOME http://setiathome.ssl.berkeley.edu/ Hunting Archibald Stewart, b 1802 in Ballymena, Antrim Co., NIR http://www.3-cities.com/~kstewart/genealogy/archibald_stewart.html To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
bind and the limit of serial number ???
Hello, is the bind have 32 bit unsigned integer variable for the serial number part of the dns records? if yes, it means that we cant have a number bigger than 4294967296 right? what happens if we have a bigger number? then bind takes it like modulus 2^32? or it is forbidden to have a bigger number? Thanks! Evren Yurtesen To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Debugging kernel data
On Sun, Apr 23, 2000 at 11:42:18AM +0930, Greg Lehey wrote: On Thursday, 13 April 2000 at 13:20:50 +0200, Mark Huizer wrote: Hi I'm trying to debug a kernel that is not crashing but hanging, with all processes in 'inode' wchan. So I did a 'call panic()', and now I have the crashdump, but is there a way to get to the data structures of the kernel??? Sure. What are you looking for? Have you read the section on kernel debugging in the handbook? Yep, done that kind of stuff before, but never on non-crashing kernels :-( Problem is I want to get at the stacks of various running processes, and the syscalls they are making. I started using the vinum gdb macros, which got me a bit further, though not yet enough, I'm afraid. Mark -- Nice testing in little China... To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: bind and the limit of serial number ???
well ours is still working fine ! thats why I asked this question, we did not realize that it went over 32 bit boundary how can I understand if there is a problem or not? Evren On Sun, 23 Apr 2000, Dave Dunaway wrote: Put a number bigger than 2^32 and it will overflows. is the bind have 32 bit unsigned integer variable for the serial number part of the dns records? if yes, it means that we cant have a number bigger than 4294967296 right? what happens if we have a bigger number? then bind takes it like modulus 2^32? or it is forbidden to have a bigger number? -- Dave. Dave Dunaway [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: bind and the limit of serial number ???
On Sun, 23 Apr 2000, Evren Yurtesen wrote: Hello, is the bind have 32 bit unsigned integer variable for the serial number part of the dns records? if yes, it means that we cant have a number bigger than 4294967296 right? Somewhere I read something like: "The format MMDDnn" is often used for the serial number. We know this wil break in the year 4294, but we are not worried about that." what happens if we have a bigger number? then bind takes it like modulus 2^32? I once put in an extra digit in the serial number. This made a secondary use a serial number, which was larger than mine, and could probably be the modulus 2^32. I had to call the hostmaster there (A "3.rd secondary" hosted at our uplink) to get the zonefile removed, so the right one would be reloaded. or it is forbidden to have a bigger number? Not only forbidden, impossible... Leif To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Help with SIOCADDMULTI, IFF_ALLMULTI and IFF_PROMISC
On Fri, 21 Apr 2000, Bill Fenner wrote: What should I do when I see ALLMULTI? You should go promiscuous to multicast. If the card doesn't support that, you should go fully promiscuous and drop unicasts that aren't to you. (Make sure you don't block packets that bpf is going to want to see). Also, if you want bridging support, you have to bridge before doing the filtering. Really, the filtering on ``appropriateness'' should only occur when the packet is about to enter ether_input(). Speaking of which, I'd really like to see bridge support getting added to all new NIC drivers from now on. However, the current bridge interface leaves something to be desired... Robert N M Watson [EMAIL PROTECTED] http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Double buffered cp(1)
:The bugs were fixed in 4.0? What I saw was far too much cpu going to :setiathome on 3.4. Seti hardly ran on 4.0. I don't have quantities :because I have only noticed the really large increase in cpu time :required to build 4.0. The wall clock time on a buildworld hardly :changed (55-60 minutes) whether I ran seti or not on 3.4. I can watch :the wall clock time on some of my future builds and see how they are I don't know what your setup is, Kent, but when I do a buildworld my system is 95% cpu bound, with virtual no idle time at all. It's all going to the buildworld. On both 3.x and 4.x. Try mounting /usr/obj with softupdates turned on, and if your /tmp is not a softupdates partition then make sure you are building with -pipe in /etc/make.conf: (For 3.x) CFLAGS= -O2 -pipe (For 4.x) CFLAGS= -Os -pipe :I was looking at this as more of a real world setup simulation. Seti :is almost pure cpu and the buildworld used everything else. I ran the :build world from an x-term and from the command line. That didn't seem :to matter much. The system also provided my dialup and nat'ing for my :internal network. Seti was run from a script that was started from an :altF2 login before I did my startx. It would chug along from one :network problem at Berkeley until the next one with out any Seti does almost no I/O, which means that in general it will have a much lower scheduling priority then even a cpu-bound buildworld. Most people run seti niced to +20, which means that it does not run at all if there are other 'normal' processes using the cpu. Under 3.x this was broken - seti got cpu when it shouldn't have. Under 4.x this is fixed. If you run a cpu-bound process niced to +20 it will get very little cpu in the face of other unniced cpu-bound processing. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: bind and the limit of serial number ???
You can not be sure your secondary dns servers are picking up your zonefile, when you update the primary. On Sun, 23 Apr 2000, Evren Yurtesen wrote: well ours is still working fine ! thats why I asked this question, we did not realize that it went over 32 bit boundary how can I understand if there is a problem or not? Evren On Sun, 23 Apr 2000, Dave Dunaway wrote: Put a number bigger than 2^32 and it will overflows. is the bind have 32 bit unsigned integer variable for the serial number part of the dns records? if yes, it means that we cant have a number bigger than 4294967296 right? what happens if we have a bigger number? then bind takes it like modulus 2^32? or it is forbidden to have a bigger number? -- Dave. Dave Dunaway [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: bind and the limit of serial number ???
:You can not be sure your secondary dns servers are picking up your :zonefile, when you update the primary. : You can 'reset' (roll-over) the serial number, but when you do so you have to realize that the secondaries may continue to cache the 'old' version of the zone file all the way up to the expiration time of the zone in the SOA (normally 3-days to a week). For manual updates of the zone file, I recommend using mmdd rather then mmddhhmm, and if you modify it twice in one day just increment the day (and hopefully real time will catch up to it again). For automatic updates (i.e. scripts that update zone files), I recommend simply starting the serial number at 1 and incrementing it on every update. Trying to make the serial number into a date for viewing ease is overrated. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: bind and the limit of serial number ???
* Leif Neland ([EMAIL PROTECTED]) [000423 13:17]: I once put in an extra digit in the serial number. This made a secondary use a serial number, which was larger than mine, and could probably be the modulus 2^32. I had to call the hostmaster there (A "3.rd secondary" hosted at our uplink) to get the zonefile removed, so the right one would be reloaded. Just FYI: if you make the serial number of a zone '0', secondary servers (bind at least) will *always* grab the zone from the master. It's intended to fix just the situation you had; set the serial to 0, leave it that way until all the slaves have picked up the new zone, then start using the proper numbering scheme again. It wastes bandwidth for a while if you have a large number of secondaries and/or a low refresh time, but it lets you fix a type without human intervention. -- j. James FitzGibbon [EMAIL PROTECTED] Targetnet.com Inc. Voice/Fax +1 416 306-0466/0452 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: bind and the limit of serial number ???
I once put in an extra digit in the serial number. This made a secondary use a serial number, which was larger than mine, and could probably be the modulus 2^32. I had to call the hostmaster there (A "3.rd secondary" hosted at our uplink) to get the zonefile removed, so the right one would be reloaded. Just FYI: if you make the serial number of a zone '0', secondary servers (bind at least) will *always* grab the zone from the master. It's intended to fix just the situation you had; set the serial to 0, leave it that way until all the slaves have picked up the new zone, then start using the proper numbering scheme again. It wastes bandwidth for a while if you have a large number of secondaries and/or a low refresh time, but it lets you fix a type without human intervention. This is a BIND feature, and should not be relied on. From RFC 1982: Caution should also be exercised before causing the serial number to be set to the value zero. While this value is not in any way special in serial number arithmetic, or to the DNS SOA serial number, many DNS implementations have incorrectly treated zero as a special case, with special properties, and unusual behaviour may be expected if zero is used as a DNS SOA serial number. There are better methods if you need to lower the serial number. The standard method, which is guaranteed to work, is to increase the serial number by 2^31-1 on the primary (largest increase allowed by serial number arithmetic, se RFC 1982), wait one refresh period (then the slave will have picked up the new serial number), and *then* set it to the desired value. Steinar Haug, Nethelp consulting, [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Double buffered cp(1)
Matthew Dillon wrote: :The bugs were fixed in 4.0? What I saw was far too much cpu going to :setiathome on 3.4. Seti hardly ran on 4.0. I don't have quantities :because I have only noticed the really large increase in cpu time :required to build 4.0. The wall clock time on a buildworld hardly :changed (55-60 minutes) whether I ran seti or not on 3.4. I can watch :the wall clock time on some of my future builds and see how they are I don't know what your setup is, Kent, but when I do a buildworld my system is 95% cpu bound, with virtual no idle time at all. It's all going to the buildworld. On both 3.x and 4.x. It is on 4.x but it wasn't on 3.4. At least, seti was taking time that it shouldn't have been getting. Try mounting /usr/obj with softupdates turned on, and if your /tmp is not a softupdates partition then make sure you are building with -pipe in /etc/make.conf: (For 3.x) CFLAGS= -O2 -pipe (For 4.x) CFLAGS= -Os -pipe I am trying this right now. I thought that optimizing this way was dangerous for the kernel. The problem for the new people is what works and when. You have to understand that Murphy sits on my shoulder. If there are 2+ bugs in a product, I will eventually end up seeing one of them :). I have been going to turn on softupdates but haven't got there yet. I want to link /usr/obj to the scsi drive first. Then, I think I will try softupdates. Try the worst combo's first and then add features but write the times down when you try the different arrangements. If 50% of my Cray throughput was write behind caching, then softupdates could be an equal share on FreeBSD. A benchmark that ran in four hours versus eight hours is a big change. I don't have a good batch system but the speed of my FreeBSD systems have a lot going for them clock wise. The Cray had 16MW (128MB) of memory and that is pretty much a starter system now days. Kent -- Kent Stewart Richland, WA mailto:[EMAIL PROTECTED] http://www.3-cities.com/~kstewart/index.html FreeBSD News http://daily.daemonnews.org/ SETI(Search for Extraterrestrial Intelligence) @ HOME http://setiathome.ssl.berkeley.edu/ Hunting Archibald Stewart, b 1802 in Ballymena, Antrim Co., NIR http://www.3-cities.com/~kstewart/genealogy/archibald_stewart.html To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Double buffered cp(1)
:
: I don't know what your setup is, Kent, but when I do a buildworld
: my system is 95% cpu bound, with virtual no idle time at all. It's
: all going to the buildworld. On both 3.x and 4.x.
:
:It is on 4.x but it wasn't on 3.4. At least, seti was taking time that
:it shouldn't have been getting.
Right. I think we're in agreement, it's just getting lost somewhere :-)
: (For 3.x)
:
: CFLAGS= -O2 -pipe
:
: (For 4.x)
:
: CFLAGS= -Os -pipe
:
:I am trying this right now. I thought that optimizing this way was
:dangerous for the kernel. The problem for the new people is what works
:and when. You have to understand that Murphy sits on my shoulder. If
:there are 2+ bugs in a product, I will eventually end up seeing one of
:them :).
-O2 and -Os are 'safe'. Other optimization levels (-O3 for example)
are *NOT* safe. Most machine-specific optimizations (-m options) should
generally be considered NOT safe.
:I have been going to turn on softupdates but haven't got there yet. I
:want to link /usr/obj to the scsi drive first. Then, I think I will
:try softupdates. Try the worst combo's first and then add features but
:write the times down when you try the different arrangements.
:...
:Kent
I also turn off atime updates ('noatime' mount option in /etc/fstab)
on /usr/src, since buildworlds do a lot of file scanning and there
is simply no reason to update every single file's inode with a new
atime. I doubt it makes much of a difference but it does leave
more clean buffer cache buffers available for other things.
-pipe makes a significant difference since without it every source
file being compiled creates several files in /tmp. Softupdates makes
a significant difference in its ability to asynchronize file creation
and meta-data updates (which helps /tmp as well as /usr/obj, though
/tmp is helped even more by using -pipe).
-Matt
Matthew Dillon
[EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Double buffered cp(1)
Matthew Dillon [EMAIL PROTECTED] wrote: -pipe makes a significant difference since without it every source file being compiled creates several files in /tmp. Hasn't O'Brien recently said that in fact "-pipe" is already the default for our cc, so explicitly specifying the option doesn't do anything? -- Christian "naddy" Weisgerber [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
commit MAKE_SHELL?
Some weeks ago, I made a small modification to make(1) so that it could
be built to use a different shell for its work. It already seemed to
have the idea at least partially thought out and in the code, so it was
a naturally easy thing to finish implementing. The code will remain
exactly the same unless you set the following (e.g. in make.conf):
MAKE_SHELL=ksh
Well, it's been sitting in my tree for a while now, and there have been
0 problems with it. I use the ports and build worlds extensively, not
to mention having built everything which uses make(1) with this. The
pdksh is more than sufficiently Bourne (or shall I say compatible? It's
just a superset...) to support every single thing I've thrown at it.
Of course, it would be possible to use the real ksh with this patch, or
whatever ksh you want named "/bin/ksh".
I certainly don't mind adding more shells to the ${MAKE_SHELL} logic, but
so far have only done ksh because using pdksh as the ${MAKE_SHELL} does,
for me, result in about 10% faster make world time, and speeds port
building enormously (though port building speed has gotten faster with
some reason optimizations made to the Mk/bsd.port*.mk files).
I'd like to get public opinion on this change to see if many others will
find it useful. Note that the change also allows you to use "/bin/csh"
with MAKE_SHELL=csh, but you might not want to shoot yourself in the foot
like that :) Let me know if you find it useful to you; if people do find
it useful enough to have these make(1) speed increases, I'll commit it
to 5.0 and 4.0. It should also apply to 3.4, but I haven't tried it yet,
as my machines run -CURRENT.
Let me know what kind of results you have :) I'm interested in results
with the real Korn shell, zsh, or bash, as well; I'll specifically add
those to the logic if people have good results there.
Thanks for any feedback!
--
Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! /
[EMAIL PROTECTED]`--'
Index: Makefile
===
RCS file: /usr2/ncvs/src/usr.bin/make/Makefile,v
retrieving revision 1.13
diff -u -r1.13 Makefile
--- Makefile1999/11/15 17:07:45 1.13
+++ Makefile2000/02/04 05:34:37
@@ -12,4 +12,20 @@
lstMember.c lstNext.c lstOpen.c lstRemove.c lstReplace.c lstSucc.c
.PATH: ${.CURDIR}/lst.lib
+# Set the shell which make(1) uses. Bourne is the default, but a decent
+# Korn shell works fine, and much faster. Using the C shell for this
+# will almost certainly break everything, but it's Unix tradition to
+# allow you to shoot yourself in the foot if you want to :-)
+
+MAKE_SHELL?= sh
+.if ${MAKE_SHELL} == "csh"
+CFLAGS+= -DDEFSHELL=0
+.elif ${MAKE_SHELL} == "sh"
+CFLAGS+= -DDEFSHELL=1
+.elif ${MAKE_SHELL} == "ksh"
+CFLAGS+= -DDEFSHELL=2
+.else
+.error "MAKE_SHELL must be set to one of \"csh\", \"sh\" or \"ksh\"."
+.endif
+
.include bsd.prog.mk
Index: config.h
===
RCS file: /usr2/ncvs/src/usr.bin/make/config.h,v
retrieving revision 1.9
diff -u -r1.9 config.h
--- config.h1999/09/10 20:51:59 1.9
+++ config.h2000/02/04 05:23:52
@@ -39,8 +39,6 @@
* $FreeBSD: src/usr.bin/make/config.h,v 1.9 1999/09/10 20:51:59 julian Exp $
*/
-#defineDEFSHELL1 /* Bourne shell */
-
/*
* DEFMAXJOBS
* DEFMAXLOCAL
Index: job.c
===
RCS file: /usr2/ncvs/src/usr.bin/make/job.c,v
retrieving revision 1.17
diff -u -r1.17 job.c
--- job.c 2000/01/17 06:43:40 1.17
+++ job.c 2000/02/03 23:24:35
@@ -191,6 +191,16 @@
"v", "e",
},
/*
+ * KSH description. The Korn shell has a superset of
+ * the Bourne shell's functionality.
+ */
+{
+"ksh",
+TRUE, "set -", "set -v", "set -", 5,
+TRUE, "set -e", "set +e",
+"v", "e",
+},
+/*
* UNKNOWN.
*/
{
Index: main.c
===
RCS file: /usr2/ncvs/src/usr.bin/make/main.c,v
retrieving revision 1.35
diff -u -r1.35 main.c
--- main.c 1999/11/23 10:35:24 1.35
+++ main.c 2000/03/25 20:02:54
@@ -483,6 +483,13 @@
/* avoid faults on read-only strings */
static char syspath[] = _PATH_DEFSYSPATH;
+#if DEFSHELL == 2
+ /*
+* Turn off ENV to make ksh happier.
+*/
+ unsetenv("ENV");
+#endif
+
#ifdef RLIMIT_NOFILE
/*
* get rid of resource limit on file descriptors
@@ -1005,7 +1012,13 @@
(void) dup2(fds[1], 1);
(void) close(fds[1]);
+#if DEFSHELL == 1
(void) execv("/bin/sh", args);
+#elif DEFSHELL == 2
+ (void) execv("/bin/ksh", args);
+#else
+#error "DEFSHELL must be 1 or 2."
Re: Help with SIOCADDMULTI, IFF_ALLMULTI and IFF_PROMISC
Robert Watson writes: You should go promiscuous to multicast. If the card doesn't support that, you should go fully promiscuous and drop unicasts that aren't to you. (Make sure you don't block packets that bpf is going to want to see). Also, if you want bridging support, you have to bridge before doing the filtering. Really, the filtering on ``appropriateness'' should only occur when the packet is about to enter ether_input(). Speaking of which, I'd really like to see bridge support getting added to all new NIC drivers from now on. However, the current bridge interface leaves something to be desired... I'm working on fixing this.. see discussion on freebsd-net. -Archie ___ Archie Cobbs * Whistle Communications, Inc. * http://www.whistle.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Debugging kernel data
On Sunday, 23 April 2000 at 18:25:28 +0200, Mark Huizer wrote: On Sun, Apr 23, 2000 at 11:42:18AM +0930, Greg Lehey wrote: On Thursday, 13 April 2000 at 13:20:50 +0200, Mark Huizer wrote: Hi I'm trying to debug a kernel that is not crashing but hanging, with all processes in 'inode' wchan. So I did a 'call panic()', and now I have the crashdump, but is there a way to get to the data structures of the kernel??? Sure. What are you looking for? Have you read the section on kernel debugging in the handbook? Yep, done that kind of stuff before, but never on non-crashing kernels :-( Problem is I want to get at the stacks of various running processes, and the syscalls they are making. I started using the vinum gdb macros, which got me a bit further, though not yet enough, I'm afraid. What are you missing? You can get a stack trace, but you'll have to go looking for the variables yourself. Greg -- Finger [EMAIL PROTECTED] for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
On Sun, 23 Apr 2000, Brian Fundakowski Feldman wrote: Some weeks ago, I made a small modification to make(1) so that it could be built to use a different shell for its work. It already seemed to have the idea at least partially thought out and in the code, so it was a naturally easy thing to finish implementing. The code will remain exactly the same unless you set the following (e.g. in make.conf): MAKE_SHELL=ksh [some eliding] in general, it's a good idea, but the problem, is, it gives a lot more freedom without any real gain. It only allows one to write Makefiles that use your private choice of shell, and you can do anything you want, I think, as it is now. Your next paragraph, where you give "csh" as an example, it (I think) a perfect counter-example, because (as we all know) csh is a horrible scripting language, but I bet it gets used. Not only that, but one will end up having hav have 15 different shells installed on machines, just to build things that require odd shells. Just so you don't have to use sh. The idea of being able to use your own shell is great, until you think about how you're opening the door to abuse of it. I'd like to get public opinion on this change to see if many others will find it useful. Note that the change also allows you to use "/bin/csh" with MAKE_SHELL=csh, but you might not want to shoot yourself in the foot like that :) Let me know if you find it useful to you; if people do find it useful enough to have these make(1) speed increases, I'll commit it to 5.0 and 4.0. It should also apply to 3.4, but I haven't tried it yet, as my machines run -CURRENT. Let me know what kind of results you have :) I'm interested in results with the real Korn shell, zsh, or bash, as well; I'll specifically add those to the logic if people have good results there. Thanks for any feedback! -- Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! / [EMAIL PROTECTED]`--' Chuck Robey| Interests include C Java programming, FreeBSD, [EMAIL PROTECTED] | electronics, communications, and signal processing. New Year's Resolution: I will not sphroxify gullible people into looking up fictitious words in the dictionary. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Multithreaded server performance
I am currently porting a multithreaded TCP server from NT (yech!) to UNIX using pthreads. The server has a fairly straightforward design -- it opens a thread for each connection, and each thread spends most of its life blocked in a call to read() from a socket. As soon as it receives enough of a request, it does quite a bit of processing and sends something back to the client. How would FreeBSD 4.0 perform in such a scenario? We are talking hundreds, maybe thousands of threads, a lot of them doing blocking reads and writes. Is the standard pthreads library adequate for the task, or would "Linuxthreads" be a better choice? What is the main difference between the standard FreeBSD pthreads and "Linuxthreads" -- it seems both are implemented using a version of clone(). The hardware is probably going to be UP at first -- a fast Pentium III (733MHz?), an Intel 820 board, an Adaptec U2W SCSI controller, and a couple of U2W LVD SCA disks to go with it. The operating system has yet to be chosen. I have tried Solaris 7 on similar hardware, and it seems so much slower than FreeBSD -- and so bloody unresponsive when doing I/O that even NT seems faster. On the other hand, Solaris's threads implementation is supposedly better than anything else out there. I'm not even considering Linux -- or should I? Is it safe to use FreeBSD 4.0-S on a production server? So far I've had nothing but positive experience with it. Any advice will be much appreciated. Alex Keahan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
I have been using it as well (with pdksh) for quite some time. I haven't
compared speed, but it certainly hasn't broken anything.
Robert Muir
[EMAIL PROTECTED]
On Sun, 23 Apr 2000, Brian Fundakowski Feldman wrote:
Some weeks ago, I made a small modification to make(1) so that it could
be built to use a different shell for its work. It already seemed to
have the idea at least partially thought out and in the code, so it was
a naturally easy thing to finish implementing. The code will remain
exactly the same unless you set the following (e.g. in make.conf):
MAKE_SHELL=ksh
Well, it's been sitting in my tree for a while now, and there have been
0 problems with it. I use the ports and build worlds extensively, not
to mention having built everything which uses make(1) with this. The
pdksh is more than sufficiently Bourne (or shall I say compatible? It's
just a superset...) to support every single thing I've thrown at it.
Of course, it would be possible to use the real ksh with this patch, or
whatever ksh you want named "/bin/ksh".
I certainly don't mind adding more shells to the ${MAKE_SHELL} logic, but
so far have only done ksh because using pdksh as the ${MAKE_SHELL} does,
for me, result in about 10% faster make world time, and speeds port
building enormously (though port building speed has gotten faster with
some reason optimizations made to the Mk/bsd.port*.mk files).
I'd like to get public opinion on this change to see if many others will
find it useful. Note that the change also allows you to use "/bin/csh"
with MAKE_SHELL=csh, but you might not want to shoot yourself in the foot
like that :) Let me know if you find it useful to you; if people do find
it useful enough to have these make(1) speed increases, I'll commit it
to 5.0 and 4.0. It should also apply to 3.4, but I haven't tried it yet,
as my machines run -CURRENT.
Let me know what kind of results you have :) I'm interested in results
with the real Korn shell, zsh, or bash, as well; I'll specifically add
those to the logic if people have good results there.
Thanks for any feedback!
--
Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! /
[EMAIL PROTECTED]`--'
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
On Sun, 23 Apr 2000, Chuck Robey wrote: in general, it's a good idea, but the problem, is, it gives a lot more freedom without any real gain. It only allows one to write Makefiles that use your private choice of shell, and you can do anything you want, I think, as it is now. That's not the purpouse, though. Yes, anyone can do that now, as they're perfectly free to modify the source as they please. And, yes, csh was in the source already (!) which is only why I put it there. Your next paragraph, where you give "csh" as an example, it (I think) a perfect counter-example, because (as we all know) csh is a horrible scripting language, but I bet it gets used. Not only that, but one will end up having hav have 15 different shells installed on machines, just to build things that require odd shells. Just so you don't have to use sh. This is more than about freedom of shells for make(1). Noone is going to write proprietary Makefiles simply because noone is going to accept the heat for trying to make people recompile their make(1) just to build their program. Don't put the idea off because of this, as I really feel that this is unrealistic. The idea of being able to use your own shell is great, until you think about how you're opening the door to abuse of it. The door's already open. Go to src/usr.bin/make. for i in *.[ch]; do perl -pi 's:/bin/sh:/bin/bash:g' $i; done And you can rebuild make(1) to use bash. It's extremely simple to make it use the shell of the day. The idea here is that our /bin/sh is not nearly as fast for many things as other Bourne shells. There should be the option to allow a user to enable a faster shell. I can easily disable the csh logic, and anyone can easily reenable it. What do you have against enabling people to use another Bourne-compatible shell if it makes things faster? I'm only criticizing the logic that it will make people want to use various shells to make Makefiles nonstandard. There's GNU make for that, anyway. Chuck Robey| Interests include C Java programming, FreeBSD, [EMAIL PROTECTED] | electronics, communications, and signal processing. New Year's Resolution: I will not sphroxify gullible people into looking up fictitious words in the dictionary. -- Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! / [EMAIL PROTECTED]`--' To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
On Sun, 23 Apr 2000, Brian Fundakowski Feldman wrote: On Sun, 23 Apr 2000, Chuck Robey wrote: in general, it's a good idea, but the problem, is, it gives a lot more freedom without any real gain. It only allows one to write Makefiles that use your private choice of shell, and you can do anything you want, I think, as it is now. [some deletions] The door's already open. Go to src/usr.bin/make. I didn't say what you asked for was hard, I was against making the addition of lots of extra complications in Makefiles too easy. To tell you the truth, I would find your idea really neat as a programmer, UNTIL I had to maintain some of the inevitable garbage that it generated among some of the less disciplined minds. That said, I'm not dead set against it, I was just giving my take. I think, overall, it's going to hugely complicate makefile maintenance, but if you get approval, go for it! There's GNU make for that, anyway. Guess why I don't like Gmake? Chuck Robey| Interests include C Java programming, FreeBSD, [EMAIL PROTECTED] | electronics, communications, and signal processing. New Year's Resolution: I will not sphroxify gullible people into looking up fictitious words in the dictionary. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
Brian Fundakowski Feldman wrote:
+.if ${MAKE_SHELL} == "csh"
+CFLAGS+= -DDEFSHELL=0
...
+#if DEFSHELL == 1
(void) execv("/bin/sh", args);
+#elif DEFSHELL == 2
+ (void) execv("/bin/ksh", args);
+#else
+#error "DEFSHELL must be 1 or 2."
+#endif
why allow csh in the Makefile when the setting of DEFSHELL it results in
will not compile? Have I missed something?
--
Ben Smithurst / [EMAIL PROTECTED] / PGP: 0x99392F7D
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Multithreaded server performance
On Mon, Apr 24, 2000 at 05:17:10AM +0300, A G F Keahan wrote: I am currently porting a multithreaded TCP server from NT (yech!) to UNIX using pthreads. The server has a fairly straightforward design -- it opens a thread for each connection, and each thread spends most of its life blocked in a call to read() from a socket. As soon as it receives enough of a request, it does quite a bit of processing and sends something back to the client. This design isn't ideal on any OS, but the fact that you do significant processing every time a request arrives on a socket probably hides most of the inefficiency due to thread switching and lack of cache locality due to many thread stacks. How would FreeBSD 4.0 perform in such a scenario? We are talking hundreds, maybe thousands of threads, a lot of them doing blocking reads and writes. Is the standard pthreads library adequate for the task, or would "Linuxthreads" be a better choice? What is the main difference between the standard FreeBSD pthreads and "Linuxthreads" -- it seems both are implemented using a version of clone(). FreeBSD's threads should perform adequately given the design of your program and the hardware you listed. Actually trying it on the various operating systems would be a good exercise, but I have found that FreeBSD's threads perform at least as well as Solaris's for such an application. LinuxThreads will definitely bog down with so many threads because the kernel scheduler has to deal with so many clone()d processes. FreeBSD's libc_r does not use clone() or anything similar. Instead, it is a userland call conversion library that multiplexes threads in a single process. This style of threads library should perform well for the type of application you are dealing with. Note that there is also ports/devel/linuxthreads, which is based on rfork(), which can be made to behave like Linux's clone(). Jason To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
On Mon, 24 Apr 2000, Ben Smithurst wrote: why allow csh in the Makefile when the setting of DEFSHELL it results in will not compile? Have I missed something? It's there to allow you attempt to shoot yourself in the foot more safely. Too much shooting, still? -- Ben Smithurst / [EMAIL PROTECTED] / PGP: 0x99392F7D -- Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! / [EMAIL PROTECTED]`--' To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Double buffered cp(1)
:Matthew Dillon [EMAIL PROTECTED] wrote: : : -pipe makes a significant difference since without it every source : file being compiled creates several files in /tmp. : :Hasn't O'Brien recently said that in fact "-pipe" is already the :default for our cc, so explicitly specifying the option doesn't do :anything? : :-- :Christian "naddy" Weisgerber [EMAIL PROTECTED] I don't recall him saying anything like that. There was an argument over making the -pipe the default a few months ago, but I was under the impression that it didn't happen because some people were concerned about the memory requirements nixing compiles on machines with small amounts of memory (since cpp, cc1, AND as wind up all running simultaniously). The documentation doesn't indicate that -pipe is turned on by default. If I ktrace a cc command it sure looks like it uses /tmp files when -pipe is not specified! So I would say that -pipe is NOT the default. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
