Re: organization
mohamed aslan wrote: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. I am driver developer, and I work with both Linux and FreeBSD. It is usual for me to changed OS I am working with a several times a day. What can I say, both source trees have some organization problems. Personally I prefer BSD one and more dislike Linux one. IMHO this is matter of taste. By the way is this your first feeling or you have some experience with BSD hacking? (e.q. try to start programming using other language or other environment, the first feeling would be the same) rik and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Wed, Mar 30, 2005, David Leimbach wrote: Yes, procfs rules! Procfs is from linux? I thought it was from Plan 9... along with rfork :). Nope. It was first implemented by Sun's Roger Faulkner in SVR4, well before Linux or Plan 9 existed. Actually, someone wrote a prototype for Unix years earlier than raf, but I don't remember who. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Thu, Mar 31, 2005 at 07:20:13AM -0500, David Schultz wrote: On Wed, Mar 30, 2005, David Leimbach wrote: Yes, procfs rules! Procfs is from linux? I thought it was from Plan 9... along with rfork :). Nope. It was first implemented by Sun's Roger Faulkner in SVR4, well before Linux or Plan 9 existed. Actually, someone wrote a prototype for Unix years earlier than raf, but I don't remember who. procfs comes from v8 (research) unix, a direct predecessor of Plan 9, way before SVR4. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Thu, Mar 31, 2005, Christoph Hellwig wrote: On Thu, Mar 31, 2005 at 07:20:13AM -0500, David Schultz wrote: On Wed, Mar 30, 2005, David Leimbach wrote: Yes, procfs rules! Procfs is from linux? I thought it was from Plan 9... along with rfork :). Nope. It was first implemented by Sun's Roger Faulkner in SVR4, well before Linux or Plan 9 existed. Actually, someone wrote a prototype for Unix years earlier than raf, but I don't remember who. procfs comes from v8 (research) unix, a direct predecessor of Plan 9, way before SVR4. That's the prototype I was talking about, but I believe it was not an official part of version 8 (to the extent that anything was). It certainly never made it to System V. Do you recall who wrote the prototype? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Thu, Mar 31, 2005 at 08:16:42AM -0500, David Schultz wrote: procfs comes from v8 (research) unix, a direct predecessor of Plan 9, way before SVR4. That's the prototype I was talking about, but I believe it was not an official part of version 8 (to the extent that anything was). It certainly never made it to System V. Do you recall who wrote the prototype? Not sure I'd call V8 a prototype ;-) See: T. J. Killian, `Processes as Files' USENIX Summer Conference Proceedings, June 1984, Salt Lake City, UT, USA for the glory details ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
i cann't reply to all of ur comments but , that is what makes u break off , as DragonFly split of u u took my opinion as an attack, u just wanna flaming, u also got off topic CVS and SVN, my words were really facts Mr Scott , Linux layout is better than FreeBSD layout , FreeBSD performance it better than Linux one , and thnx for silly reply. that's why i hate forums and maillists and i should mail this directly to the core members. On Tue, 29 Mar 2005 16:46:04 -0600, Craig Boston [EMAIL PROTECTED] wrote: At the risk of going further and further off-topic from freebsd-hackers... On Tue, Mar 29, 2005 at 02:29:13PM -0800, Bruce A. Mah wrote: Sounds like a bad situation there. On our server we use svn+ssh, except for a few Windows clients that use https. (BTW our server is running 4-STABLE and it's wonderful.) Hmmm, I initially didn't want to use that because I read that it suffers from the same security issues as CVS. The appeal of being able to fine-tune permissions and grant subversion access without shell access is quite luring. HTTP timeouts during long operations, on the other hand, suck. ( my server is woefully underpowered :-D ). Note to davsvn users with slow servers: http-timeout = 3600 is your friend. Heh. :-) 1.1.3 is current now, but one can find mentions of a 1.1.4 bugfix release being planned, as well as the (farther out) 1.2 release with locking. Oh, I've been running 1.1.3 on both client and server since it went into ports (many dump/loads later). Just haven't taken the time to see what's new and compare to older versions. :) Craig ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] -- I'm Searching For Perfection, So Even If U Need Portability U've To Use Assembly ;-) http://www.maslanlab.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
Le Mercredi 30 mars 2005 à 09:30 -0800, mohamed aslan a écrit :
i cann't reply to all of ur comments
but , that is what makes u break off , as DragonFly split of u
u took my opinion as an attack,
u just wanna flaming,
*shrug*
u also got off topic CVS and SVN,
That's a long time topic.
my words were really facts Mr Scott , Linux layout is better than
FreeBSD layout , FreeBSD performance it better than Linux one , and
thnx for silly reply.
You {c,sh}ould have been flamed a lot for such a message. After
reading your three messages, I'm not sure that's not what you
were looking for.
If you want to talk about facts, don't say I think that
that's why i hate forums and maillists and i should mail this directly
to the core members.
I'm quite sure you'll get the same answer.
--
Florent Thoumie
[EMAIL PROTECTED]
signature.asc
Description: This is a digitally signed message part
Re: organization
On Wed, 30 Mar 2005 09:30:47 -0800, mohamed aslan [EMAIL PROTECTED] wrote: i cann't reply to all of ur comments but , that is what makes u break off , as DragonFly split of u So you're going to fork FreeBSD because we didn't think your comments were constructive and that you don't like the organization of FreeBSD's sources? Ok, have fun :). u took my opinion as an attack, u just wanna flaming, u also got off topic CVS and SVN, You have to keep in mind that what you intended may not come through with the way your email reads. Popping into a forum and saying this sucks is a sure fire way to get flamed ANYWHERE. You should have provided some reasoning why you believed FreeBSD's source layout was not as good as Linux's from whatever perspective you were looking at it. my words were really facts Mr Scott , Linux layout is better than FreeBSD layout , FreeBSD performance it better than Linux one , and thnx for silly reply. FreeBSD performance is better than Linux. That's yet another interesting claim. For the applications I run no one would touch FreeBSD as it doesn't do as well as Linux so that's a very general and very vague claim. that's why i hate forums and maillists and i should mail this directly to the core members. So stop posting to them. On Tue, 29 Mar 2005 16:46:04 -0600, Craig Boston [EMAIL PROTECTED] wrote: At the risk of going further and further off-topic from freebsd-hackers... On Tue, Mar 29, 2005 at 02:29:13PM -0800, Bruce A. Mah wrote: Sounds like a bad situation there. On our server we use svn+ssh, except for a few Windows clients that use https. (BTW our server is running 4-STABLE and it's wonderful.) Hmmm, I initially didn't want to use that because I read that it suffers from the same security issues as CVS. The appeal of being able to fine-tune permissions and grant subversion access without shell access is quite luring. HTTP timeouts during long operations, on the other hand, suck. ( my server is woefully underpowered :-D ). Note to davsvn users with slow servers: http-timeout = 3600 is your friend. Heh. :-) 1.1.3 is current now, but one can find mentions of a 1.1.4 bugfix release being planned, as well as the (farther out) 1.2 release with locking. Oh, I've been running 1.1.3 on both client and server since it went into ports (many dump/loads later). Just haven't taken the time to see what's new and compare to older versions. :) Craig ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] -- I'm Searching For Perfection, So Even If U Need Portability U've To Use Assembly ;-) http://www.maslanlab.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Wed, 2005-Mar-30 09:30:47 -0800, mohamed aslan wrote: u took my opinion as an attack, Your phrasing was provocative (though at least you agree that it's just your opinion - elsewhere, you continue to claim that your opinions are facts). u just wanna flaming, Given your statements, I was surprised at how restrained people were. u also got off topic CVS and SVN, Many people can see areas where the FreeBSD layout is less than ideal. One major stumbling block to global change is CVS - which can't really handle renaming files. CVS's shortcomings are widely known and there are regular side-discussions on what (if anything) to do about changing. Since you proposed a major rename, it was reasonable to discuss how this could be achieved. my words were really facts Mr Scott , Linux layout is better than FreeBSD layout , This is a very subjective statement. The only evidence you've provided to justify it is that an ls on the top level kernel source directory is shorter on Linux than FreeBSD - without explaining why the short list is better. If you are serious about the Linux layout being better, how about proposing a new layout, providing a list of rules that define what files go where (so that someone with a new kernel file to add can identify where it goes), explaining how your new layout is better than the current layout and providing a mapping for all current file locations to new file locations. FreeBSD performance it better than Linux one , Whilst performance can be measured, it's impossible to define overall system performance, irrespective of the hardware or task, as a single number that can be compared between two systems. thnx for silly reply. I saw very few of those. that's why i hate forums and maillists and i should mail this directly to the core members. I'm not sure this is the best way to endear yourself to the FreeBSD community. This strikes me as a good way to get yourself added to personal killfiles. -- Peter Jeremy ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
mohamed aslan wrote: guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. Nope. It's real difficult to organize the files worse than in Linux. FreeBSD is actually real good. Way better than UnixWare, and of course anything beats Linux with its crazy patchwork. primary os. but we can get the good things from linux and through out the bad ones. Yes, procfs rules! -SB ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
Yes, procfs rules! Procfs is from linux? I thought it was from Plan 9... along with rfork :). -SB ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Tue, 2005-Mar-29 10:50:32 +0300, Giorgos Keramidas wrote: A file may be repo-copied to a new location, then removed from the HEAD branch in the old location and deleted from the rest of the branches in the new location. This way the history will be there, in both places but the file will only 'live' in one place at a time. At the cost of approximately doubling the repository size - ncvs/src/sys is currently abut 366MB. There are an awful lot of repository copies lying around and even at current disk prices, the total cost in dollars is non-trivial. The bigger cost is developer time - kernel developers are familiar with the current re-organisation. It will probably take at least a month for a developer to get used to a new organisation. Multiply that by the number of developers and there's an awful lot of lost productive time. The new layout would have to offer really good incentives to have a net benefit. -- Peter Jeremy ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
The biggest problem is keeping history here. Doing something like that with CVS is a major PITA. We didn't have any old release, so moving the repository files didn't create a problem. That's impossible in FreeBSD land :) wasnt here some discussion about moving FreeBSD to subversion (as some other projects did - samba, mono etc.)? and subversion solves this... roman ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . if ls the files in freebsd sys subdir , u will got about 54 subdirs and a makefile while linux contains about 15 subdirs only. guys, don't take my words against bsd , i admit that the performace of bsd is much better than linux and i'm planning to change to it as my primary os. but we can get the good things from linux and through out the bad ones. On Mon, 28 Mar 2005 18:39:14 -0300, Patrick Tracanelli [EMAIL PROTECTED] wrote: Hey there Mohamed, hello. I did not take it as a flame war initiative nor reason for such a thing. This kind of consideration is good to make people think about some points which are rarerely discussed. I personally disagree w/ you but it is also my very personal opinion. Anyway, this is not the reason I am emailing you now. You reply came to me only, since the list does not change the reply-to header you should retype the address or Reply All. I think your points should go on the list, so resend it there if it is the case. I am considering you really intended to reply to the list. If its not the case and you wanted to reply only to me, do not consider this message. Thanks, Bye :-) mohamed aslan wrote: guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . if ls the files in freebsd sys subdir , u will got about 54 subdirs and a makefile while linux contains about 15 subdirs only. guys, don't take my words against bsd , i admit that the performace of bsd is much better than linux and i'm planning to change to it as my primary os. but we can get the good things from linux and through out the bad ones. On Mon, 28 Mar 2005 16:26:12 -0300, Patrick Tracanelli [EMAIL PROTECTED] wrote: Maybe you are just more familiar to Linux kernel. I am not a kernel hacker, like you and many people here. But I usually read source codes, FreeBSD and also NetBSD and Linux, specially the areas where I am a particular curious. FreeBSD code organization is close to BSD's roots (you can get those Walnut Creek historical CDROM which has code for 4BSD and 386BSD to compare). I like FreeBSD orgaization better. Maybe you will disagree it for a thousand years, or one day find NetBSD approach better than both. In any case I am sure spending more time under FreeBSD's src/ won't make the organization such a deal that deserves this comment. mohamed aslan wrote: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. -- I'm Searching For Perfection, So Even If U Need Portability U've To Use Assembly ;-) http://www.maslanlab.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
From: mohamed aslan [EMAIL PROTECTED] Subject: Re: organization Date: Tue, 29 Mar 2005 07:41:25 -0800 guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . The problem is getting consensus on what is to be done. Sure, one can arbitrarily say this goes here or that goes there, but everyone's notion of reorg is a little different. It would take a lot of time and energy to get this consensus, which is better spent making things work better... Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
mohamed aslan wrote: guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . if ls the files in freebsd sys subdir , u will got about 54 subdirs and a makefile while linux contains about 15 subdirs only. guys, don't take my words against bsd , i admit that the performace of bsd is much better than linux and i'm planning to change to it as my primary os. but we can get the good things from linux and through out the bad ones. Have you ever thought that time is better spent on engineering rather than silly dicussions of 'facts' like yours? I really don't give a hoot what Linux's layout is, nor whether you think that changing FreeBSD to be more like it is a trivial task. Scott ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
Warner Losh wrote: From: mohamed aslan [EMAIL PROTECTED] Subject: Re: organization Date: Tue, 29 Mar 2005 07:41:25 -0800 guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . The problem is getting consensus on what is to be done. Sure, one can arbitrarily say this goes here or that goes there, but everyone's notion of reorg is a little different. It would take a lot of time and energy to get this consensus, which is better spent making things work better... Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] And the worts of all is that You are both right to some extent. The new developers want the source tree arranged in the way mohamed says it should be. Not some device drivers live in pci/ other in dev/ and things like that. And on the other hand experienced kernel hackers who want things to stay as they are so it is easy for them to navigate in know waters. IMHO mohamed is a bit more right. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
Iasen Kostov wrote: Warner Losh wrote: From: mohamed aslan [EMAIL PROTECTED] Subject: Re: organization Date: Tue, 29 Mar 2005 07:41:25 -0800 guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . The problem is getting consensus on what is to be done. Sure, one can arbitrarily say this goes here or that goes there, but everyone's notion of reorg is a little different. It would take a lot of time and energy to get this consensus, which is better spent making things work better... Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] And the worts of all is that You are both right to some extent. The new developers want the source tree arranged in the way mohamed says it should be. Not some device drivers live in pci/ other in dev/ and things like that. And on the other hand experienced kernel hackers who want things to stay as they are so it is easy for them to navigate in know waters. IMHO mohamed is a bit more right. What do you think the reaction would be if you or Mr. Aslan trotted over to the linux kernel hackers list and told them that they were all 'wrong' for piling all of the storage drivers into linux/drivers/scsi instead of separating them out into subdirectories? What do you think the reaction would be if you told Theo DeRadt that OpenBSD is 'wrong' for piling dozens of drivers into sys/dev/ic instead of separating them out more logically? At best, you'd be ignored. Telling a group that they are right or wrong based on your personal opinion of how the world should be is, um, a waste of time. Scott ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Tue, Mar 29, 2005, Warner Losh wrote: From: mohamed aslan [EMAIL PROTECTED] Subject: Re: organization Date: Tue, 29 Mar 2005 07:41:25 -0800 guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . The problem is getting consensus on what is to be done. Sure, one can arbitrarily say this goes here or that goes there, but everyone's notion of reorg is a little different. It would take a lot of time and energy to get this consensus, which is better spent making things work better... I think few people would disagree with certain changes, like putting MD bits in subdirectories called 'arch' as in NetBSD. The real question is whether people care enough to justify the repo bloat and the extra load on the cvsup mirrors. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
At 7:41 AM -0800 3/29/05, mohamed aslan wrote: guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. for example all fs has to go in a subdir called fs arch specific file should go in subdir called arch/(arch name) and so on . One thing to watch out for is the mess this would cause in the CVS repository. CVS does not track file moves, so if we move a lot of things around then we just end up with them in *both* the old and the new locations. I certainly believe the tree could be organized better than it is, but the benefits from reorganizing are just not worth the time and effort it would take (*), and the mess it would make out of the CVS repository. (* - 99% of the time and effort would be in getting everyone to *agree* on the best layout...) -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
In message: [EMAIL PROTECTED] David Schultz [EMAIL PROTECTED] writes: : On Tue, Mar 29, 2005, Warner Losh wrote: : From: mohamed aslan [EMAIL PROTECTED] : Subject: Re: organization : Date: Tue, 29 Mar 2005 07:41:25 -0800 : : guys this is not a flame war : but the linux way in arranging the source file is really better than : freebsd way, it's a fact. : however it's easy to rearrange it in 1 min as someone said before. : but i mean this step should be done from the core team. : for example all fs has to go in a subdir called fs : arch specific file should go in subdir called arch/(arch name) : and so on . : : The problem is getting consensus on what is to be done. Sure, one can : arbitrarily say this goes here or that goes there, but everyone's : notion of reorg is a little different. It would take a lot of time : and energy to get this consensus, which is better spent making things : work better... : : I think few people would disagree with certain changes, like : putting MD bits in subdirectories called 'arch' as in NetBSD. The : real question is whether people care enough to justify the repo : bloat and the extra load on the cvsup mirrors. You've proven my point exactly: Some folks want to see i386 moved to arch/i386, others think it is stupid to do that. Discussion isn't possible here, so nothing will happen since there's no compelling reason to do anything, just a weak argument about how things might be nicer. The fact that we even consider cvsup load when discussing this means that clearly it is a weak idea: if we have to worry about the impact on how we distribute the sources for a change, isn't that really a weird criteria to use? Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
[EMAIL PROTECTED] wrote: And the worts of all is that You are both right to some extent. The new developers want the source tree arranged in the way mohamed says it should be. Not some device drivers live in pci/ other in dev/ and things like that. And on the other hand experienced kernel hackers who want things to stay as they are so it is easy for them to navigate in know waters. IMHO mohamed is a bit more right. If you are really dedicated to studying the kernel sources at that level issues such as learning the layout are really nonissues, the time a new developer initially spends hunting around for files because they are not familiar with the layout is insignificant compared to the time experienced developers would lose as a result of a new layout being introduced, especially during a critical development stage. Once all the major changes are finished and we move to micro optimization a new, more consistent and logical layout would be an option, but until then, IMHO, things should stay the way they are. Also, the benefit of such a reorganization at this time does not justify the work needed to make it happen. It Would Be Nice (TM), but it's not practical. ALeine ___ WebMail FREE http://mail.austrosearch.net ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
In message: [EMAIL PROTECTED] Divacky Roman [EMAIL PROTECTED] writes: : The biggest problem is keeping history here. Doing something like that : with CVS is a major PITA. We didn't have any old release, so moving : the repository files didn't create a problem. That's impossible in : FreeBSD land :) : : wasnt here some discussion about moving FreeBSD to subversion (as some other : projects did - samba, mono etc.)? and subversion solves this... Subversion was deemed to be insufficiently mature for FreeBSD last time it was evaluated formally. Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Tue, Mar 29, 2005, M. Warner Losh wrote: In message: [EMAIL PROTECTED] David Schultz [EMAIL PROTECTED] writes: : On Tue, Mar 29, 2005, Warner Losh wrote: : From: mohamed aslan [EMAIL PROTECTED] : Subject: Re: organization : Date: Tue, 29 Mar 2005 07:41:25 -0800 : : guys this is not a flame war : but the linux way in arranging the source file is really better than : freebsd way, it's a fact. : however it's easy to rearrange it in 1 min as someone said before. : but i mean this step should be done from the core team. : for example all fs has to go in a subdir called fs : arch specific file should go in subdir called arch/(arch name) : and so on . : : The problem is getting consensus on what is to be done. Sure, one can : arbitrarily say this goes here or that goes there, but everyone's : notion of reorg is a little different. It would take a lot of time : and energy to get this consensus, which is better spent making things : work better... : : I think few people would disagree with certain changes, like : putting MD bits in subdirectories called 'arch' as in NetBSD. The : real question is whether people care enough to justify the repo : bloat and the extra load on the cvsup mirrors. You've proven my point exactly: Some folks want to see i386 moved to arch/i386, others think it is stupid to do that. Discussion isn't possible here, so nothing will happen since there's no compelling reason to do anything, just a weak argument about how things might be nicer. The fact that we even consider cvsup load when discussing this means that clearly it is a weak idea: if we have to worry about the impact on how we distribute the sources for a change, isn't that really a weird criteria to use? Indeed, both the pro and con arguments are weak, which is probably why nothing has happened. I for one would love to see libm called libm and not msun, for instance, but when it comes down to it, I have better things to do. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Tue, Mar 29, 2005 at 05:05:38PM +0200, Divacky Roman wrote: wasnt here some discussion about moving FreeBSD to subversion (as some other projects did - samba, mono etc.)? and subversion solves this... Yes, a few people have looked at it from time to time (raises hand as one of the guilty parties). The last I heard, subversion did not scale well to the massive amount of files that are in the FreeBSD repository. IIRC it's been a while since this was tested, so it may or may not be true anymore. SVK may partially address this by bypassing libwc. Also, repository size is a big issue (no pun intended). If adding a few hundred megs for repo-copies is prohibitively expensive, I don't think increasing the repo size by many gigabytes would go over very well. Subversion repositories can easily be several times the size of a CVS repository containing the same data. Craig ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
[EMAIL PROTECTED] wrote: You've proven my point exactly: Some folks want to see i386 moved to arch/i386, others think it is stupid to do that. Discussion isn't possible here, so nothing will happen since there's no compelling reason to do anything, just a weak argument about how things might be nicer. If such a layout change were to happen I think the least time wasting procedure would go something like this: - issue a request for new layout propositions - for a week accept proposition submissions (web based) - have the FreeBSD community vote on the proposed layouts (web based) - assign a deadline by which a decision has to be made and then discuss the most popular layouts and try to get the Core Team and the committers to reach a consensus No consensus, no change. In any case, a lot of time and energy would be spent discussing this change, it's more of a bike shed issue than some might think, it's just not worth even starting this process in the near future. The fact that we even consider cvsup load when discussing this means that clearly it is a weak idea: if we have to worry about the impact on how we distribute the sources for a change, isn't that really a weird criteria to use? It does not prove that the idea itself is weak, it's a criterion that proves that CVS is not well suited to such changes. The thing is that even if such a change would be trivial to implement with no additional overhead, the process of deciding which new layout to adopt would take too much time and energy compared to the benefits gained by adopting a new layout, at least at this stage of development. ALeine ___ WebMail FREE http://mail.austrosearch.net ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
mohamed aslan wrote this message on Tue, Mar 29, 2005 at 07:41 -0800: Also, learn not to top post... it looses context... guys this is not a flame war but the linux way in arranging the source file is really better than freebsd way, it's a fact. well, as I stated in a previous email, if you make a statement without detailing your reasons, many people feel that you are attacking FreeBSD.. You should be impressed that you didn't get more flames... however it's easy to rearrange it in 1 min as someone said before. but i mean this step should be done from the core team. No, this step should only be done by the repo manager... and let the people who have extensive experience handle it.. in some cases, it means that we'll have to keep two copies of the files for a long time so that all branches are properly buildable, or we'll have to go back and change all the old branches to make sure they use the new location.. That's a large undertaking... for example all fs has to go in a subdir called fs well, not even NetBSD does this, and they are better organized than FreeBSD is... but luckily, you can just do ls -d *fs and get them... Though the reasons that ufs and nfs and isofs aren't in the fs is hysterical raisins... It's been a while since I looked at this, but I was surprised how many were in the fs dir.. so, this is more of a minor point.. arch specific file should go in subdir called arch/(arch name) and so on . if ls the files in freebsd sys subdir , u will got about 54 subdirs and a makefile while linux contains about 15 subdirs only. Again, FreeBSD was originally only i386 (and pc64), and for a long time, only a two arch project.. it wasn't till a few years ago that we started to grow many new arches (amd64, ia64, sparc64, powerpc, arm, and others).. So, at the time, putting i386 (and pc98 and alpha) at the top level made sense, but now that we have so many, yes, it doesn't make sense, but at the same time the cost (as others have layed out) is expensive to do Also, as time permits, such as when new drivers are written, old drivers retires, drivers rewritten, things are improving... such as the dev dir, instead of putting the drivers in isa, they are moved into dev.. one example is sio.c.. that used to be an isa device driver, but now lives in dev/sio + the various bus attachments since it is no longer isa specific... guys, don't take my words against bsd , i admit that the performace of bsd is much better than linux and i'm planning to change to it as my primary os. but we can get the good things from linux and through out the bad ones. Or at least the ideas.. :) we can't take in too much GPL code into the tree.. then it'd just be pointless... :) -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Tue, 29 Mar 2005 12:17:53 -0500 David Schultz [EMAIL PROTECTED] wrote: On Tue, Mar 29, 2005, M. Warner Losh wrote: In message: [EMAIL PROTECTED] David Schultz [EMAIL PROTECTED] writes: : On Tue, Mar 29, 2005, Warner Losh wrote: : From: mohamed aslan [EMAIL PROTECTED] : Subject: Re: organization : Date: Tue, 29 Mar 2005 07:41:25 -0800 : : guys this is not a flame war : but the linux way in arranging the source file is really better than : freebsd way, it's a fact. : however it's easy to rearrange it in 1 min as someone said before. : but i mean this step should be done from the core team. : for example all fs has to go in a subdir called fs : arch specific file should go in subdir called arch/(arch name) : and so on . : : The problem is getting consensus on what is to be done. Sure, one can : arbitrarily say this goes here or that goes there, but everyone's : notion of reorg is a little different. It would take a lot of time : and energy to get this consensus, which is better spent making things : work better... : : I think few people would disagree with certain changes, like : putting MD bits in subdirectories called 'arch' as in NetBSD. The : real question is whether people care enough to justify the repo : bloat and the extra load on the cvsup mirrors. You've proven my point exactly: Some folks want to see i386 moved to arch/i386, others think it is stupid to do that. Discussion isn't possible here, so nothing will happen since there's no compelling reason to do anything, just a weak argument about how things might be nicer. The fact that we even consider cvsup load when discussing this means that clearly it is a weak idea: if we have to worry about the impact on how we distribute the sources for a change, isn't that really a weird criteria to use? Indeed, both the pro and con arguments are weak, which is probably why nothing has happened. I for one would love to see libm called libm and not msun, for instance, but when it comes down to it, I have better things to do. Equivalent (or nearly equivalent) gains could probably be made by simply documenting the current layout better. Also, that's the sort of project that someone like Mohamed could undertake with minimal contention from the rest of the project. -Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
If memory serves me right, Craig Boston wrote:
On Tue, Mar 29, 2005 at 05:05:38PM +0200, Divacky Roman wrote:
wasnt here some discussion about moving FreeBSD to subversion (as some other
projects did - samba, mono etc.)? and subversion solves this...
Yes, a few people have looked at it from time to time (raises hand as
one of the guilty parties).
The last I heard, subversion did not scale well to the massive amount of
files that are in the FreeBSD repository. IIRC it's been a while since
this was tested, so it may or may not be true anymore. SVK may
partially address this by bypassing libwc.
Well, someone's part-way there with a Subversion mirror of src/. From
http://www.freebsd.org/support.html:
A public Subversion mirror of the FreeBSD src/ CVS repository is
provided at svn://svn.clkao.org/freebsd/. A web interface is also
available. This is intended for people who would like to try the svk
distributed version control system.
This of course doesn't include ports/ or doc/, so it doesn't really
answer the scalability question.
Also, repository size is a big issue (no pun intended). If adding a few
hundred megs for repo-copies is prohibitively expensive, I don't think
increasing the repo size by many gigabytes would go over very well.
Subversion repositories can easily be several times the size of a CVS
repository containing the same data.
This is dependent (among other things) on the nature of the files in the
repository and which repository back-end is used. I did a conversion at
${REALJOB} in December where I converted 1.3GB of CVS repository to
about 1.5GB in Subversion. For the curious, the back-end was FSFS, and
an earlier test conversion using the BDB back-end took about 2.1GB. I
know this is smaller than the FreeBSD repository.
I have this vague, handwavy feeling (colored no doubt by positive
experiences using it at ${REALJOB}) that Subversion, as well as the
conversion tool (cvs2svn) have matured a bit since the last time this
topic came up. But I'm not necessarily advocating a switch either.
Cheers,
Bruce.
signature.asc
Description: This is a digitally signed message part
Re: organization
On Tue, Mar 29, 2005 at 11:22:19AM -0600, Craig Boston wrote: The last I heard, subversion did not scale well to the massive amount of files that are in the FreeBSD repository. IIRC it's been a while since this was tested, so it may or may not be true anymore. SVK may partially address this by bypassing libwc. That's not true. There are two major problems with subversion, compared to CVS: - the size of the working copy is doubled (because of the local cache) - annotation is linear in the number of revisions (of a file?) The first can be work-arounded by using SVK, but often is also an advantage, because e.g. diff is a pure local operation which doesn't have to contact the server. The second is related to how subversion stores the data. There are some persons working on speeding it up by using a cache, but I'm not sure how far the work is. On the other hand, CVS definitely doesn't scale to large repositories too, just think about the time a cvs up or cvs tag needs. You can't make everything fast, it is a compromise between speed, disk space and not to forget atomarity. Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Tue, Mar 29, 2005 at 01:25:19PM -0800, Bruce A. Mah wrote:
Well, someone's part-way there with a Subversion mirror of src/. From
http://www.freebsd.org/support.html:
A public Subversion mirror of the FreeBSD src/ CVS repository is
provided at svn://svn.clkao.org/freebsd/.
Ah, yes, I do remember Chia-liang Kao was working on a CVS -
Subversion bridge for use with svk. The ability to do incremental
updates, even if it's only one-way, is a big win on both sides of the
fence.
This of course doesn't include ports/ or doc/, so it doesn't really
answer the scalability question.
Most of what I ran into was just in src/. I hesitate to say anything
since it's been a long time and my memory is vague. ISTR a few simple
operations on a file at the top level were causing the entire tree to be
traversed. Unfortunately I don't remember exactly -- maybe it's time to
test it again...
One issue I do remember had to more do with apache and davsvn rather
than subversion itself. Attempting to cancel a long running operation
(say an accidental svn diff of the whole tree) would kill the frontend
but leave the backend churning away on the server, which would bog down
further requests (locking?), causing them to hang for a while, build up
requests, and make the situation worse.
I use subversion exclusively for my personal projects but none are big
enough that I've run into that problem again. It may well have been
fixed by now -- the version number has been creeping up while I wasn't
looking :)
This is dependent (among other things) on the nature of the files in
the repository and which repository back-end is used. I did a
conversion at ${REALJOB} in December where I converted 1.3GB of CVS
repository to about 1.5GB in Subversion. For the curious, the
back-end was FSFS, and an earlier test conversion using the BDB
back-end took about 2.1GB. I know this is smaller than the FreeBSD
repository.
Ah, I haven't played with FSFS yet. All my repositories are BDB that
have been upgraded and migrated from version 0.something.
Craig
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Tue, Mar 29, 2005 at 11:34:11PM +0200, Joerg Sonnenberger wrote: That's not true. There are two major problems with subversion, compared to CVS: - the size of the working copy is doubled (because of the local cache) - annotation is linear in the number of revisions (of a file?) Not trying to spread false information about Subversion -- I like it very much and use it for all my personal projects :) Just stating my opinion based on observations made while using it. The first can be work-arounded by using SVK, but often is also an advantage, because e.g. diff is a pure local operation which doesn't have to contact the server. Well, you don't have the extra working copy files, but you usually end up eating up at least as much space for your local repository mirror; unless you have a lot of working copies checked out. On the other hand, CVS definitely doesn't scale to large repositories too, just think about the time a cvs up or cvs tag needs. You can't make everything fast, it is a compromise between speed, disk space and not to forget atomarity. Keeping in mind that the tests I ran were back in the pre-1.0 days (right before 1.0 IIRC), Subversion was much faster on update, but significantly slower for checkout and diffs. Those operations scaled worse than O(n) as the repository grew. It would be interesting to run some more benchmarks (clkao's mirror eliminates much of the import hassle) and see how the latest subversion compares. Also, as Bruce reminded me, the new fsfs storage backend may have different characteristics that need to be taken into account. Of course Subversion fares much better on the atomicity issue, that's a given :) svk may be able to help too. I tried it for a while but eventually gave up because getting the perl bindings installed and working was a bit of a black art. Probably time to try the port again. Craig ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
If memory serves me right, Craig Boston wrote:
On Tue, Mar 29, 2005 at 01:25:19PM -0800, Bruce A. Mah wrote:
This of course doesn't include ports/ or doc/, so it doesn't really
answer the scalability question.
Most of what I ran into was just in src/. I hesitate to say anything
since it's been a long time and my memory is vague. ISTR a few simple
operations on a file at the top level were causing the entire tree to be
traversed. Unfortunately I don't remember exactly -- maybe it's time to
test it again...
I know what you mean...I was looking through my notes for more details
about our repository conversion and they were a little lacking. :-p
One issue I do remember had to more do with apache and davsvn rather
than subversion itself. Attempting to cancel a long running operation
(say an accidental svn diff of the whole tree) would kill the frontend
but leave the backend churning away on the server, which would bog down
further requests (locking?), causing them to hang for a while, build up
requests, and make the situation worse.
Sounds like a bad situation there. On our server we use svn+ssh, except
for a few Windows clients that use https. (BTW our server is running
4-STABLE and it's wonderful.)
I use subversion exclusively for my personal projects but none are big
enough that I've run into that problem again. It may well have been
fixed by now -- the version number has been creeping up while I wasn't
looking :)
Heh. :-) 1.1.3 is current now, but one can find mentions of a 1.1.4
bugfix release being planned, as well as the (farther out) 1.2 release
with locking.
This is dependent (among other things) on the nature of the files in
the repository and which repository back-end is used. I did a
conversion at ${REALJOB} in December where I converted 1.3GB of CVS
repository to about 1.5GB in Subversion. For the curious, the
back-end was FSFS, and an earlier test conversion using the BDB
back-end took about 2.1GB. I know this is smaller than the FreeBSD
repository.
Ah, I haven't played with FSFS yet. All my repositories are BDB that
have been upgraded and migrated from version 0.something.
While BDB worked well for us in testing, FSFS gave us the ability to do
incremental backups of the repository, which was important for getting
buy-in from my IT support group. I was a little nervous at using new
code for the backend but it's had nary a hiccup. (Knock on wood.)
As you probably know, a number of people have reported lockups with the
BDB backend...it turns out to be more problems with the way that
Subversion uses BDB, as well as people just not setting it up correctly.
Bruce.
signature.asc
Description: This is a digitally signed message part
Re: organization
At the risk of going further and further off-topic from freebsd-hackers... On Tue, Mar 29, 2005 at 02:29:13PM -0800, Bruce A. Mah wrote: Sounds like a bad situation there. On our server we use svn+ssh, except for a few Windows clients that use https. (BTW our server is running 4-STABLE and it's wonderful.) Hmmm, I initially didn't want to use that because I read that it suffers from the same security issues as CVS. The appeal of being able to fine-tune permissions and grant subversion access without shell access is quite luring. HTTP timeouts during long operations, on the other hand, suck. ( my server is woefully underpowered :-D ). Note to davsvn users with slow servers: http-timeout = 3600 is your friend. Heh. :-) 1.1.3 is current now, but one can find mentions of a 1.1.4 bugfix release being planned, as well as the (farther out) 1.2 release with locking. Oh, I've been running 1.1.3 on both client and server since it went into ports (many dump/loads later). Just haven't taken the time to see what's new and compare to older versions. :) Craig ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
organization
hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
mohamed aslan wrote this message on Mon, Mar 28, 2005 at 10:01 -0800: it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. If you are going to provide criticism, you should provide some reasons.. and/or examples of why... It's like me saying that the Linux kernel build infastructure isn't very clean... and then giving no reason for it... but I'm not going to be like you, and give you a reason: I can't speek for current versions of Linux, but older ones if you changed a small part of your kernel config, it would require you to rebuild the entire kernel source tree... With FreeBSD, I can add or remove drivers, and my new kernel will be done in about a minute... Much nicer... Please provide more content in your messages to this list... -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 28 Mar 2005, mohamed aslan wrote: hi guys it's my first post here, BTW i was a Linux hacker and Linux kernel mailing list member for 3 years. and I've a comment here , i think the freebsd kernel source files aren't well organized as Linux ones. Well thats nice, Didnt your mommy say once to you before If you dont have nothing good to say then dont say nothing ?. Thats not a really detailed look into where you think the files are not well organized. Personly I think you have had your head stuck in the Linux culture to long to comment on anything going on with the Freebsd code or how the system is layed out. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (FreeBSD) Comment: http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xF7DF979F iD8DBQFCSFALsmFQuvffl58RAgB5AJ43Fjog004aDzcKZMiEPw5DVQSTzACbB4R+ eo+3aE+Bq+JDmEWksXkYhLo= =8AHQ -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Mon, Mar 28, 2005 at 01:42:16PM -0500, c0ldbyte wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 28 Mar 2005, mohamed aslan wrote: hi guys it's my first post here, BTW i was a Linux hacker and Linux kernel mailing list member for 3 years. and I've a comment here , i think the freebsd kernel source files aren't well organized as Linux ones. Well thats nice, Didnt your mommy say once to you before If you dont have nothing good to say then dont say nothing ?. Thats not a really detailed look into where you think the files are not well organized. Personly I think you have had your head stuck in the Linux culture to long to comment on anything going on with the Freebsd code or how the system is layed out. Perhaps you should take a bit of your own advice `c0ldbyte'. I'm personally sick of your condescending attitude towards everybody on the public lists. Cut it out. --Devon pgpNJ6xL4OHDT.pgp Description: PGP signature
Re: organization
mohamed aslan wrote: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. You are in some ways correct.. Unfortunatly, as our project is significantly older than Linux it has a lot more historical baggage.. The system layout has its roots in teh layout of the original Unix layout from the 1970s. The BSD source tree has lived through several complete generations of hardware. In the beginning for example, IO was one of 5 things.. DISCs (expensive) tape, Teletype (110baud paper printer) or card readers and punches. As things have changed, some of the original layout decisions have become rather outdated. For a slightly better example, check out the layout of the DragonflyBSD kernel sources. Matt took the oportunity to re-arange the FreeBSD sources when he imported them.. To some extent I agree with him (though not necessariy with his positioning of every file). It is possible that we could do with a reoganisation but it isn't a work-free job.. Matt took some time to get everything working again.. possible things that could be done.. Every driver in its own directory. INCLUDING the makefile for the module and the man pages.. All network protocols to be sunk into a networking subdirectory ditto for filesystems. Bus drivers being separated slightly from device drivers (though related (/sys/dev/bus?/usb?) The kern directory broken into functional sub parts. (proces control, scheduler, ipc etc) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
c0ldbyte wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 28 Mar 2005, mohamed aslan wrote: hi guys it's my first post here, BTW i was a Linux hacker and Linux kernel mailing list member for 3 years. and I've a comment here , i think the freebsd kernel source files aren't well organized as Linux ones. Well thats nice, Didnt your mommy say once to you before If you dont have nothing good to say then dont say nothing ?. Thats not a really detailed look into where you think the files are not well organized. Personly I think you have had your head stuck in the Linux culture to long to comment on anything going on with the Freebsd code or how the system is layed out. now now.. don't snap his head off for that.. it's a reasonable comment for someone who is maybe lookign to do work in both places.. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (FreeBSD) Comment: http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xF7DF979F iD8DBQFCSFALsmFQuvffl58RAgB5AJ43Fjog004aDzcKZMiEPw5DVQSTzACbB4R+ eo+3aE+Bq+JDmEWksXkYhLo= =8AHQ -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
Devon H. O'Dell wrote: On Mon, Mar 28, 2005 at 01:42:16PM -0500, c0ldbyte wrote: On Mon, 28 Mar 2005, mohamed aslan wrote: hi guys it's my first post here, BTW i was a Linux hacker and Linux kernel mailing list member for 3 years. and I've a comment here , i think the freebsd kernel source files aren't well organized as Linux ones. Well thats nice, Didnt your mommy say once to you before If you dont have nothing good to say then dont say nothing ?. Thats not a really detailed look into where you think the files are not well organized. Personly I think you have had your head stuck in the Linux culture to long to comment on anything going on with the Freebsd code or how the system is layed out. Perhaps you should take a bit of your own advice `c0ldbyte'. I'm personally sick of your condescending attitude towards everybody on the public lists. Cut it out. Fully seconded. Even though it clearly wasn't a brilliant idea to criticize FreeBSD source code organization without giving a single reason or example, Mohamed clearly didn't deserve to be flamed like this. A bit more diplomacy wouldn't hurt here... Now responding to Mohamed, I'm really surprised to see you writing this. I've banged my head against the wall so many times to locate some piece of source code in the Linux kernel that I really cannot understand you. For the most part, I find the FreeBSD source organization simply great. There are many criticisms one can legitimally do against FreeBSD, but that one sounds just wrong. Cheers, Maxime ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Mon, Mar 28, 2005 at 11:05:30AM -0800, Julian Elischer wrote: As things have changed, some of the original layout decisions have become rather outdated. For a slightly better example, check out the layout of the DragonflyBSD kernel sources. Matt took the oportunity to re-arange the FreeBSD sources when he imported them.. To some extent I agree with him (though not necessariy with his positioning of every file). I completely agree here, but it is difficult to get everything into the perfect place. The NetBSD idea of moving i386 and the other platforms into arch/ is also very nice. It is possible that we could do with a reoganisation but it isn't a work-free job.. Matt took some time to get everything working again.. The biggest problem is keeping history here. Doing something like that with CVS is a major PITA. We didn't have any old release, so moving the repository files didn't create a problem. That's impossible in FreeBSD land :) Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
Maybe you are just more familiar to Linux kernel. I am not a kernel hacker, like you and many people here. But I usually read source codes, FreeBSD and also NetBSD and Linux, specially the areas where I am a particular curious. FreeBSD code organization is close to BSD's roots (you can get those Walnut Creek historical CDROM which has code for 4BSD and 386BSD to compare). I like FreeBSD orgaization better. Maybe you will disagree it for a thousand years, or one day find NetBSD approach better than both. In any case I am sure spending more time under FreeBSD's src/ won't make the organization such a deal that deserves this comment. mohamed aslan wrote: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. Oh, a Linux list member for 3 years. Wow! and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. And I think you are wrong and that the Linux sources a tangled mess. This sort of whining isn't productive. Why don't you think they are organized? Because the organization is different than Linux? Or some other reason? Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
The user mohamed aslan wrote this strange object...: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] Well, If I have to give my point, I think that FreeBSD has a very nice one, because the things are easy to find in it. I never tried any Linux, only FreeBSD. But you say that it's not organized well, like linux ones, You can discuss this for ages, because you used linux for years and we do *BSD for years so we can say that the linux ones aren't organized. It just depends wat your used to. But everyone has his own point about what he likes to use. Only thing I don't like at FreeBSD is the wine, I need to run Wine with some programs, Like internet explorer but it just dont want to install, followed some tutorials but nothing.. well if you might think you can solve this, please contact me privately I really want to fix this :S even the freebsd-emulators and wine mailinglists doesn't help :( Regards, JeanPaul (Nexohrion) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On Mon, 28 Mar 2005 23:49:25 +0200 (CEST), Webmaster Shizukana.net [EMAIL PROTECTED] wrote: The user mohamed aslan wrote this strange object...: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] Well, If I have to give my point, I think that FreeBSD has a very nice one, because the things are easy to find in it. I never tried any Linux, only FreeBSD. But you say that it's not organized well, like linux ones, You can discuss this for ages, because you used linux for years and we do *BSD for years so we can say that the linux ones aren't organized. It just depends wat your used to. But everyone has his own point about what he likes to use. Only thing I don't like at FreeBSD is the wine, I need to run Wine with some programs, Like internet explorer but it just dont want to install, followed some tutorials but nothing.. well if you might think you can solve this, please contact me privately I really want to fix this :S even the freebsd-emulators and wine mailinglists doesn't help :( Regards, JeanPaul (Nexohrion) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] Seems to me people waste a lot of time replying to stupid emails. The best solution is to ignore. Too bad I didn't follow my own advice. Dave ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
The user David Leimbach On Mon, 28 Mar 2005 23:49:25 +0200 (CEST), Webmaster Shizukana.net [EMAIL PROTECTED] wrote: The user mohamed aslan wrote this strange object...: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] Well, If I have to give my point, I think that FreeBSD has a very nice one, because the things are easy to find in it. I never tried any Linux, only FreeBSD. But you say that it's not organized well, like linux ones, You can discuss this for ages, because you used linux for years and we do *BSD for years so we can say that the linux ones aren't organized. It just depends wat your used to. But everyone has his own point about what he likes to use. Only thing I don't like at FreeBSD is the wine, I need to run Wine with some programs, Like internet explorer but it just dont want to install, followed some tutorials but nothing.. well if you might think you can solve this, please contact me privately I really want to fix this :S even the freebsd-emulators and wine mailinglists doesn't help :( Regards, JeanPaul (Nexohrion) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] Seems to me people waste a lot of time replying to stupid emails. The best solution is to ignore. Too bad I didn't follow my own advice. Dave Well that's what you do if you are bored... Or if you are irritated because wine doesn't function proper and it's you're own fault because you can't configure it... (That's what happened to me) -Nexohrion ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
mohamed aslan wrote: hi guys it's my first post here, BTW i was a linux hacker and linux kernel mailing list member for 3 years. and i've a comment here , i think the freebsd kernel source files aren't well organized as linux ones. I left a job last year where I developed linux kernel code professionally for almost 5 years. When I first looked at the linux tree, it took some time to get used to how it was layed out, but eventually it came naturally to my fingers. I've also used and developed *BSD kernel code for 12 years, so I know that layout too. And now I'm getting my hands dirty in the OSX/Darwin sources. To tell you the truth, it doesn't make a difference either way. If your concern with a software project is how the source tree is layed out, then I highly doubt we'll be seeing much contribution from you. Thanks. Scott ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: organization
On 2005-03-28 21:17, Joerg Sonnenberger [EMAIL PROTECTED] wrote: On Mon, Mar 28, 2005 at 11:05:30AM -0800, Julian Elischer wrote: As things have changed, some of the original layout decisions have become rather outdated. For a slightly better example, check out the layout of the DragonflyBSD kernel sources. Matt took the oportunity to re-arange the FreeBSD sources when he imported them.. To some extent I agree with him (though not necessariy with his positioning of every file). I completely agree here, but it is difficult to get everything into the perfect place. The NetBSD idea of moving i386 and the other platforms into arch/ is also very nice. It is possible that we could do with a reoganisation but it isn't a work-free job.. Matt took some time to get everything working again.. The biggest problem is keeping history here. Doing something like that with CVS is a major PITA. We didn't have any old release, so moving the repository files didn't create a problem. That's impossible in FreeBSD land :) Not impossible. Just extremelly annoying, cumbersome and error-prone. A file may be repo-copied to a new location, then removed from the HEAD branch in the old location and deleted from the rest of the branches in the new location. This way the history will be there, in both places but the file will only 'live' in one place at a time. I go agree though that the work this requires for thousands of files is an immense task, not to be taken lightly. - Giorgos ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
New gcc organization in DFly
DFly has incorporated a new organizational structure for gcc3 and
binutils214 that FreeBSD might want to take a look at.
Basically it works like this:
* The vendor code in /usr/src/contrib is named after the major version
release, so e.g. we have /usr/src/contrib/gcc-3.3 (gcc-3.3-20040126 at
the moment) and /usr/src/contrib/binutils-2.14
* The vendor code in contrib is unmodified, with only the addition of
README files describing the setup and the deletion of all original
vendor files that the build does not require.
* The build is represented by /usr/src/gnu/usr.bin/{binutils214,cc3},
/usr/src/gnu/lib/{gcc2,gcc3}, and so forth. In DFly, the gcc2 stuff
was left in bintuils/ and cc/.
* The intention is so that we can (A) have multiple revs installed on
the system simultaniously and (B) physically remove obsolete revs from
the contrib/ portion of the repository, just leaving the build portion
in /usr/src/gnu in the repository. If a developer wants to revert
to an obsolete rev he can simply download the original vendor tar
and unpack it into contrib without modification for his local use.
* binutils-2.14 installs in /usr/libexec/binutils214/{ldscripts,elf,aout}
* gcc-3.3 installs in /usr/libexec/gcc3 and /usr/lib/gcc{2,3} (e.g. the
crt*.o, libgcc, and other version-specific files get a subdirectory
in /usr/lib).
* The front-end code, /usr/src/usr.bin/objformat, is made to understand
the new layout as well as a new environment variable, CCVER, and
exec's the appropriate binary from the appropriate location.
* installworld and/or upgrade Makefile's were modified to remove the
old binaries.
I'm not entirely finished with it yet. The gcc-3.3 build still hardwires
the binutils path, but generally speaking this methodology has worked out
very well and I am going to start applying it to other contrib stuff
in the DFly hierarchy. It makes updating vendor code a whole lot easier..
a matter of a few minutes rather then a few hours, and allows us to
maintain very new or experimental vendor code in parallel with the
production version without effecting the existing user base. And being
able to physically delete obsolete revs from the repository is a big plus
too because modified vendor code creates the worst bloat in a CVS
repository. The original impetus for the work was so that we could
integrate support for other compilers (e.g. TenDRA, ICC) as well as
future versions of gcc, rather then as port hacks. We haven't done that
yet but we now have the infrastructure to be able to.
FreeBSD might want to review this new methodology for gcc, bintuils,
and other vendor material (heimdal, openssh, kerberos, openssl, etc).
I've only managed to convert gcc and binutils so far, and it was a lot
of work to clean up the build tree in /usr/src/gnu/xxx, but it's work
that only needed to be done once and much of it is directly transportable
to the FBsd infrastructure.
-Matt
Matthew Dillon
[EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
Hi Bruce,
A few thoughts on your API:
1) Rather than naming the struct's as l1, l2 etc, it may be more
orthogonal to use an array of cache entries like so
struct entry { ... } entries[MAX_ENTRIES]; where MAX_ENTRIES would be say,
8.
2) We could pass information back about whether the cache is write-back or
write-through and whether it uses write-allocate. In some CPUs (e.g.
the AMD K6-2) this aspect of the cache is programmable at boot time.
3) Have a bit indicating whether the cache is indexed virtually or physically.
This allows us to describe TLBs and caches using the same descriptor; the
MIPS R4K used virtually addressed L1 caches, IIRC.
4) For caches and TLBs that support variable line/page sizes, we would
be reporting the currently programmed size (the kernel knows this
information) I guess?
The 'type' field of the cache descriptor could be an `u_int32_t' or
`u_int16_t',
allocated out as follows:
kind: tlb/cache/other 2 bits
addressing: virtual/physical/unknown2 bits
mode: data/instruction/both/unknown 2 bits
distance: L0/L1/L2/whatever 3 bits
on-write-hit: write-back/write-thru/unknown 2 bits
on-write-miss: write-allocate/unknown 2 bits
Another suggestion I have is that the sysctl return:
int n_entries;
struct entry entries[n_entries];
since it isn't clear how many levels of cache and how many kinds
of TLBs are going to be used in the systems of tomorrow.
Regards,
Koshy
[EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
Bruce M Simpson wrote this message on Mon, Oct 13, 2003 at 20:32 +0100: i386 pc98 amd64 --- Action: Add code to identcpu.c to fill out hw_cacheinfo. Cache discovery: Extended CPUID. Static tables if 486-class machine. No cache on 386. TLB discovery: Extended CPUID. Static tables if 486-class machine. No cache on 386. not to be a stick, but I do have a Am386DX-40 that has 128kb of cache on the mother board. It's the same style SRAMs that most 486's have on their board. -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Sun, Oct 12, 2003 at 08:57:52PM +0100, Bruce M Simpson wrote: [ Andrew: Perhaps you can shed some light on how the necessary information can be gathered on Alpha? My search was incomplete and I could not find a reliable source for DEC's development manuals. ] L1 cache information is in the CPU datasheets. I don't know of a summary across the whole Alpha family. The datasheets can be (nominally) found at: http://h18000.www1.hp.com/products/software/alpha-tools/documentation/current/chip-docs.html Last time I went digging, some of the links didn't work but if you look at the links and rummage around the FTP site, the information was all there (and other material that wasn't referenced in the HTML pages). sysctl is a good interface for retrieving this information as it doesn't change during the lifetime of the kernel, and it is small. sysctl is already invoked from within libc to retrieve information in this way. I agree. sysctl would appear to be the best interface. alpha - Cache discovery? Static. AFAIK, there's no PALcode interface, unfortunately. i386 pc98 amd64 --- Cache discovery? CPUID. Earlier chips which don't support it probably don't have a cache, or aren't worth supporting. 80386 has no on-chip cache. Intel i486 has 8KB _unified_ 4-way, 16 bytes/line L1. Cache alignment has a significant effect and gcc defaults to 16-byte alignment on -m486. ports/benchmarks/lmbench includes tools that can experimentally determine the cache configuration - though not quickly/efficiently enough to form part of the boot. Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
Peter Jeremy wrote: On Sun, Oct 12, 2003 at 08:57:52PM +0100, Bruce M Simpson wrote: [ Andrew: Perhaps you can shed some light on how the necessary information can be gathered on Alpha? My search was incomplete and I could not find a reliable source for DEC's development manuals. ] L1 cache information is in the CPU datasheets. I don't know of a summary across the whole Alpha family. The datasheets can be (nominally) found at: http://h18000.www1.hp.com/products/software/alpha-tools/documentation/current/chip-docs.html Last time I went digging, some of the links didn't work but if you look at the links and rummage around the FTP site, the information was all there (and other material that wasn't referenced in the HTML pages). sysctl is a good interface for retrieving this information as it doesn't change during the lifetime of the kernel, and it is small. sysctl is already invoked from within libc to retrieve information in this way. I agree. sysctl would appear to be the best interface. alpha - Cache discovery? Static. AFAIK, there's no PALcode interface, unfortunately. i386 pc98 amd64 --- Cache discovery? CPUID. Earlier chips which don't support it probably don't have a cache, or aren't worth supporting. 80386 has no on-chip cache. Intel i486 has 8KB _unified_ 4-way, 16 bytes/line L1. Cache alignment has a significant effect and gcc defaults to 16-byte alignment on -m486. Only the DX, SX, DX2, SX2 and GX - DX4 has a 16kB one, and it may be write through or write back. However, I believe the DX4s have CPUID so detecting them should be simple. ports/benchmarks/lmbench includes tools that can experimentally determine the cache configuration - though not quickly/efficiently enough to form part of the boot. Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
All, Here are detailed design documents for determining cache and TLB geometry across our currently supported processor architectures, with recommendations outlined for implementation. What I haven't addressed yet is how indirect consumers of the API might use it, e.g. mutex consumers vs. UMA, in the context of allocating cache-aligned mutexes from a mutex pool. Please let me know your thoughts. BMS Detailed design for cache/tlb geometry discovery all --- Review NetBSD's uvm_page_recolor() viz applicability to FreeBSD VM/UMA. alpha - Action: Add code to machdep.c in identifycpu() to fill out hw_cacheinfo. Cache discovery: Static tables keyed on specific CPU model. TLB discovery: Static tables keyed on specific CPU model. Cache heuristic: 8Kb L1 Split Direct Mapped (21064) 2MB L2 Unified Direct Mapped (21064) All CPUs below 21264 have a 32-byte L1 line size. 21264 (EV6) has a 64-byte L1 line size. Optional L3 cache. TLB heuristic: ITLB 8KB page 8 lines, 4MB page 4 lines (21064) DTLB 32 lines, all page sizes, fully associative. (21064) ia64 Action: Add code to machdep.c in identifycpu() to fill out hw_cacheinfo. Review Linux's pal.h and palinfo.c. files. Cache discovery: Call the platform functions PAL_CACHE_SUMMARY and PAL_CACHE_INFO to get this information. TLB discovery: Static tables keyed on specific CPU model. Cache heuristic: L1 typically split 4-way set-associative 16KB, L2 256KB unified, L3 3MB-6MB unified. Line size isn't defined by the architecture. TLB heuristic: L1 TLB, split, data/instruction 32 entries each, fully associative L2 TLB, split, data/instruction 128 entries each, fully associative i386 pc98 amd64 --- Action: Add code to identcpu.c to fill out hw_cacheinfo. Cache discovery: Extended CPUID. Static tables if 486-class machine. No cache on 386. TLB discovery: Extended CPUID. Static tables if 486-class machine. No cache on 386. Cache heuristic (Intel): L1: 4-way, 32 bytes/line Cache heuristic (AMD): L2: 8-way, 64 bytes/line TLB heuristic (Intel): 4KB Code: 32 entries, 4-way, LRU 4MB Code: 2 entries, Fully associative, LRU 4KB Data: 64 entries, 4-way, LRU 4MB Data: 8 entries, 4-way, LRU TLB heuristic (AMD): 4KB L1 Code: 16 entries, Fully associative, LRU 4MB/2MB L1 Code: 8 entries, Fully associative, LRU 4KB L1 Data: 24/32 entries, Fully associative, LRU 4MB/2MB L1 Data: 8 entries, 4-way, LRU 4KB L2 Code: 256 entries, 4-way, LRU 4KB L2 Data: 256 entries, 4-way, LRU (That's 6 distinct TLBs to deal with on AMD-based i386 architectures). powerpc --- Action: Adapt from NetBSD as appropriate. Cache discovery: Open Firmware on CHRP if available. Static tables keyed on specific CPU model. TLB discovery: Open Firmware on CHRP if available. Static tables keyed on specific CPU model. Cache heuristic: L1 line size: 32 bytes across family. Pre-G5: 32KB/32KB Split, 8-way G5: 64KB/32KB Split, 1-way L2 line size: 32/64/128 bytes, TLB heuristic: PPC 601e: 4KB Instruction TLB, 4 entries, most recently used translations UTLB, 256 entries, 2-way set associative, software selectable block size OFW properties: i-cache-size i-cache-sets i-cache-block-size d-cache-size d-cache-sets d-cache-block-size tlb-size tlb-sets l2-cache [*] CHRP only mips Action: Adapt from NetBSD as appropriate. Cache discovery: Static tables keyed on specific CPU model. TLB discovery: MIPS32/MIPS64 Privileged Resource Architecture registers Cache heuristic: Split/unified L1/L2, unified L3. TLB heuristic: 16KB page size, 64 entries, fully associative (R1) sparc64 --- Action: Adapt existing code in cache.c to fill out and use hw_cacheinfo. Review assembly code, particularly that which abuses the TLB. Work closely with jake@ to avoid code churn. Cache discovery: Open Firmware. TLB discovery: Open Firmware. Cache heuristic: Split L1, Unified L2. TLB heuristic: Split L1 TLB. Fully Associative. NLU. 64 lines each. OFW properties: icache-size icache-line-size icache-associativity dcache-size dcache-line-size dcache-associativity ecache-size ecache-line-size ecache-associativity #dtlb-entries #itlb-entries Maintain information about cache and TLB geometry in an MI structure. The abstraction is intended to reflect current and future machine architectures. It is expected that the contents of these structures may not change over the lifetime of the kernel. Keeping this information in a structure doesn't significantly increase the cost of retrieving it from userland anyway. Userland consumers such as thread libraries and memory allocators should take a copy of this structure upon initialization. Kernel consumers may feel free to cache the information in local variables as they like. TLBs are 'caches' for virtual address lookups. Like data and instruction caches, they may employ set associativity to reduce the risk of unnecessary cache flushes/misses in multiprogramming
Re: Determining CPU features / cache organization from userland
Hi, ISTR that AMD 486 had different cache arrangements from Intel. Just threw one out - I'll see if I can find another around here. -- Bob Bishop +44 (0)118 977 4017 [EMAIL PROTECTED] fax +44 (0)118 989 4254 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
All, I came up with the attached text file today to summarize some of my findings, after looking at various open source trees to see how they handle run-time cache geometry detection. Many will find it ironic that i386 is the easiest platform to deal with. [ Andrew: Perhaps you can shed some light on how the necessary information can be gathered on Alpha? My search was incomplete and I could not find a reliable source for DEC's development manuals. ] Jeff Roberson suggested I adopt NetBSD's API, however, on further examination it's clear that NetBSD's approach isn't consistent across all platforms. Darwin takes a similar approach, but it is perhaps too PowerPC-centric. sysctl is a good interface for retrieving this information as it doesn't change during the lifetime of the kernel, and it is small. sysctl is already invoked from within libc to retrieve information in this way. glibc's approach to dealing with situations where knowledge of the cache line size is needed is a bit fractious - it retrieves the information from an 'aux vector' passed to glibc at startup. I think threading libraries should seriously consider becoming consumers of the API once it's finalized. Mutex alignment on cache line boundaries is desirable for userland applications too. However, phk malloc would need to be changed in order to support this specific form of aligned allocation. Perhaps a separate pool or zone could be used for this kind of allocation? This becomes more important and timely when one considers the I/O alignment restrictions we've encountered. Some applications may need to align their buffers on arbitrary boundaries to suit devices, too. BMS all --- NetBSD cache information API(s) are not consistent across platforms. alpha - Cache discovery? Static. 21064, 21064A, 21066, 21066A, 21164 all have line sizes of 32-bytes. The 21264 has a 64-byte line size. 21364: L1 split, 64KB each, 2-way set-associative, Virtual caches can be implemented using PALcode, but this is probably more of a curiosity than anything else. ia64 Cache discovery? Call PAL_CACHE_INFO, I think. No documentation on how to do this at this time. I have emailed [EMAIL PROTECTED] asking for advice. i386 pc98 amd64 --- Cache discovery? CPUID. Earlier chips which don't support it probably don't have a cache, or aren't worth supporting. General rule for x86: split L1, unified L2, optional unified L3. General rule for Intel P5: 2-way, 32 bytes/line General rule for Intel MMX and up: 4-way, 32 bytes/line PPro doesn't have L3. The newer cores have different cache geometry. powerpc --- Cache line discovery? Static. Many core variants. I have not seen any runtime code for this. The POWER clcs instruction is obsolete. OpenDarwin assumes 32-bytes. It has hooks for discovering the cache geometry at runtime but these are not used. NetBSD statically initializes this information according to the discovered CPU model in use, which is the way to go. NetBSD tells uvm to recolor the page queues if required. Linux uses static #define's from IBM people, except in the case of ppc64, which is strikingly similar to the OpenDarwin code except it actually talks to the open firmware. Open Firmware on CHRP should however provide the following for each cpu device node configured in the system: i-cache-size i-cache-sets i-cache-block-size d-cache-size d-cache-sets d-cache-block-size tlb-size tlb-sets l2-cache All are integers except for l2-cache which is the address of an l2-cache device node if the system found one. mips The NetBSD MIPS code for dealing with cache geometry was recently updated. MIPS caches may be split/unified at L1/L2 and unified at L3. Cache detection code is quite voluminous. Swipe NetBSD's if FreeBSD/mips ever kicks off. Many, many core variants. sparc64 --- Cache line discovery? Performed by Open Firmware. Open Firmware property names used are ever so slightly different from Apple's. icache-size icache-line-size icache-associativity dcache-size dcache-line-size dcache-associativity ecache-size ecache-line-size ecache-associativity Already handled within cache.c, but assembly stubs *expect* this information in a certain format. Specifically they need to see the data cache/instruction cache sizes and line sizes. General rule: Split L1, Unified L2. Cores: Spitfire/Blackbird/Cheetah ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Sat, Oct 11, 2003 at 01:58:27PM +1000, Peter Jeremy wrote: If you do this, it may make sense to use the same names as MacOSX. What if your hardware has different linesizes for different caches? I noticed whilst peering in Apple Developer Notes that G5 has 128 byte cache line size, and this screws up mutexes bigtime. (!!) OS X definitions considered too PowerPC centric. I think the best way to handle all cases is thus:- - Support 3 levels of cache. - Each level may be unified or split between code and data not-quite-Von-Neumann-style. - Allow explicit retrieval of this info keyed on the cache you're interested in. This means: hw.cache.lN.(linesize|lines|sets) - Do similar for the TLB insofar as we can return information about the chip's TLB. I know for example from talking to peter@ that the Opteron is quite a different beast (ASNs, flush filter, etc). - Assume that all CPUs have identical characteristics in an SMP system. Trying to assume otherwise is pointless. People should be using matched chips anyway. BMS ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Sat, Oct 11, 2003 at 09:27:11AM +0100, Bruce M Simpson wrote: OS X definitions considered too PowerPC centric. I think the best way to handle all cases is thus:- - Support 3 levels of cache. Out of interest, do any systems other than the big-iron Alpha's use L3 cache? A quick look at the code suggests that only L2 is coloured. - Each level may be unified or split between code and data not-quite-Von-Neumann-style. Do any systems use split L2 (or L3) caches? And how do you define the wierd micro-instruction cache used in the P4? - Allow explicit retrieval of this info keyed on the cache you're interested in. This means: hw.cache.lN.(linesize|lines|sets) How do you distinguish between a direct-mapped and fully-associative cache? (Do any current CPUs have fully-associative caches?) For set-associative caches, is it worth identifying and reporting the replacement algorithm (eg random, LRU or pseudo-LRU) - Do similar for the TLB insofar as we can return information about the chip's TLB. I know for example from talking to peter@ that the Opteron is quite a different beast (ASNs, flush filter, etc). This is possibly more useful on the RISC CPUs where the TLB is managed in firmware (eg Alpha PALcode) so TLB misses are expensive. Note that at least the Alpha has multiple sets of TLB registers for different mapping types and sizes. The number of registers in each set varies between different AXP generations (though I think the sets remain the same). - Assume that all CPUs have identical characteristics in an SMP system. Trying to assume otherwise is pointless. People should be using matched chips anyway. HP AlphaServer ES47 (and ES45 from memory) allow different speed CPUs in an SMP system. Some of the high-end SPARCservers probably do as well. This probably does make sense when you're talking about a system which might be expanded over its lifetime - and the slow CPUs that came with the system initially might no longer be available but you need more CPUs and can't justify replacing the existing ones. Whether FreeBSD wants to support this market is another issue. Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Sat, Oct 11, 2003 at 08:12:31PM +1000, Peter Jeremy wrote:
Out of interest, do any systems other than the big-iron Alpha's use L3
cache? A quick look at the code suggests that only L2 is coloured.
L3 cache is present on many MIPS and Pentium Xeon systems, as well as
PowerPC G4.
Do any systems use split L2 (or L3) caches? And how do you define the
wierd micro-instruction cache used in the P4?
I believe certain models of MIPS may have split L2. Most L3 caches I
believe will be unified.
How do you distinguish between a direct-mapped and fully-associative
cache? (Do any current CPUs have fully-associative caches?) For
set-associative caches, is it worth identifying and reporting the
replacement algorithm (eg random, LRU or pseudo-LRU)
Add a sysctl type. enum cachetype { notpresent, direct, setassoc,
fullyassoc }. Only look at sets if cache type set accordingly.
[TLB]
This is possibly more useful on the RISC CPUs where the TLB is managed
in firmware (eg Alpha PALcode) so TLB misses are expensive. Note that
at least the Alpha has multiple sets of TLB registers for different
mapping types and sizes. The number of registers in each set varies
between different AXP generations (though I think the sets remain the
same).
I know a number of individuals and organizations involved with FreeBSD pay
very close attention to this, to the point of doing TLB profiling to ensure
they don't churn too much in time-critical code, particulary on i386 derived
platforms. I think knowledge of TLB geometry is valuable everywhere, but more
so in the cases you point out. sparc64 has software-managed TLB.
[on non-symmetric SMP processor clock-speeds and cache organisation]
Whether FreeBSD wants to support this market is another issue.
We'll build that bridge when we come to it.
BMS
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Determining CPU features / cache organization from userland
Hi -hackers, I'm looking for ways that a userland program can determine the CPU features available on an SMP machine -- processor model, stepping numbers, supported features, cache organization etc. For example, on some x86 processors the CPUID instruction could be used to determine some of these parameters, but using this instruction in an SMP context is a little tricky since we do not know which CPU gets to execute the instruction. Would you know of any existing APIs, in use in other OSes, for retrieving this kind of information? Regards, Koshy [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Fri, 10 Oct 2003, Joseph Koshy wrote: Hi -hackers, I'm looking for ways that a userland program can determine the CPU features available on an SMP machine -- processor model, stepping numbers, supported features, cache organization etc. For example, on some x86 processors the CPUID instruction could be used to determine some of these parameters, but using this instruction in an SMP context is a little tricky since we do not know which CPU gets to execute the instruction. At least in the Intel world, multiprocessor systems are _always_ supposed to have matching processor steppings, so the reliability of the information should be very good indeed. Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Fri, Oct 10, 2003 at 03:36:40AM -0700, Joseph Koshy wrote: I'm looking for ways that a userland program can determine the CPU features available on an SMP machine -- processor model, stepping numbers, supported features, cache organization etc. What Silby said and have a look at the sysutils/x86info port. I've been thinking we should definitely make the cache organization info available via sysctl. I am thinking we should do this to make the UMA_ALIGN_CACHE definition mean something... I will probably throw diffs Jeff's way soon for this but I'm recovering from a bit of a nasty cold right now. BMS ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Fri, Oct 10, 2003 at 02:44:00PM +0100, Bruce M Simpson wrote: On Fri, Oct 10, 2003 at 03:36:40AM -0700, Joseph Koshy wrote: I'm looking for ways that a userland program can determine the CPU features available on an SMP machine -- processor model, stepping numbers, supported features, cache organization etc. What Silby said and have a look at the sysutils/x86info port. Hey, cool, I'd never heard about this. Just tried this, and got some wierdness. Can I ask about it here, or do I poke at the port maintainer? I've been thinking we should definitely make the cache organization info available via sysctl. I am thinking we should do this to make the UMA_ALIGN_CACHE definition mean something... I will probably throw diffs Jeff's way soon for this but I'm recovering from a bit of a nasty cold right now. BMS ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] -- Brian 'you Bastard' Reichert[EMAIL PROTECTED] 37 Crystal Ave. #303Daytime number: (603) 434-6842 Derry NH 03038-1713 USA BSD admin/developer at large ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
Bruce M Simpson writes: I've been thinking we should definitely make the cache organization info available via sysctl. I am thinking we should do this to make the UMA_ALIGN_CACHE definition mean something... If you do this, it may make sense to use the same names as MacOSX. Eg: g51% sysctl hw | grep cache hw.cachelinesize: 128 hw.l1icachesize: 65536 hw.l1dcachesize: 32768 hw.l2cachesize: 524288 Drew ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Fri, Oct 10, 2003 at 03:09:47PM -0400, Andrew Gallatin wrote: Bruce M Simpson writes: I've been thinking we should definitely make the cache organization info available via sysctl. I am thinking we should do this to make the UMA_ALIGN_CACHE definition mean something... If you do this, it may make sense to use the same names as MacOSX. Eg: g51% sysctl hw | grep cache hw.cachelinesize: 128 hw.l1icachesize: 65536 hw.l1dcachesize: 32768 hw.l2cachesize: 524288 Er, that's weird, considering POWER has the CLCS instruction which is intended to support variable cache line sizes. Doesn't POWER4 and POWER5 have a cache which is split in this way? Also can we assume they are the same for all CPUs in an SMP system? I'd like to think that that is the case. BMS ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Determining CPU features / cache organization from userland
On Fri, Oct 10, 2003 at 03:09:47PM -0400, Andrew Gallatin wrote: Bruce M Simpson writes: I've been thinking we should definitely make the cache organization info available via sysctl. I am thinking we should do this to make the UMA_ALIGN_CACHE definition mean something... If you do this, it may make sense to use the same names as MacOSX. g51% sysctl hw | grep cache hw.cachelinesize: 128 hw.l1icachesize: 65536 hw.l1dcachesize: 32768 hw.l2cachesize: 524288 What if your hardware has different linesizes for different caches? Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Memory organization in case of large amount of data (jumbo grams)
Hi, My doubt is whether freebsd uses the normal mbuf clusters in case of large amount of data (like jumbogram in ipv6 or the maximum ipv4 datagram size of 65536 bytes)? My understanding is that for such a large amount of data, clusters which can hold only 2048 byes are not economical. Hence freebsd might be using some other type of buffers which have large capacity. Kindly mail me regarding this. Is my understanding correct. Kindly make me updated on buffers used in such cases. regards ravi prasad __ Get your own FREE, personal Netscape Webmail account today at http://webmail.netscape.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Mbuf organization in case of large amount of data.
Hi, My doubt is how data will be organized in buffers in case we want to transmit large amount of data. My doubt is regarding the organization of mbufs in case we want to transmit the maximum ip datagram size. In the normal case data is stored in clusters for data size greater than 208 bytes. My doubt is how the data will be organized in case of such large data. If mbufs along with clusters are used we will have a long chain. Whether any other buffers are used? how the data is organized in the buffers in such a case? How the data will be organized in case we want to transmit jumbograms in ipv6. I think that cluster concept of 2K bytes is too small. regards ravi prasad __ Get your own FREE, personal Netscape Webmail account today at http://webmail.netscape.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Memory organization in case of large amount of data (jumbo grams)
On Tue, May 15, 2001 at 08:15:56AM -0400, [EMAIL PROTECTED] wrote: My doubt is whether freebsd uses the normal mbuf clusters in case of large amount of data (like jumbogram in ipv6 or the maximum ipv4 datagram size of 65536 bytes)? FreeBSD provides two standard types of storage (mbufs and mbuf clusters) for network use, but data can be stored in other ways too - some of the gigabit ethernet drivers define jumbo clusters of size 8kB, allowing jumbo eithernet packets to be recieved in a single chunk. In general the idea of the mbuf system is that no copies of the data need to be done, so even if the data is a little fragmented it shouldn't be a big problem. (Some ethernet cards may need the data to be unfragmented before sending, but most of the good cards shouldn't. Also the important size here is not so much the IPv6 of IPv4 maximum sizes, but the size of the MTU on the medium on which you are tramsmittin.) David. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
