Re: Glibc-based Debian GNU/KNetBSD
(Sorry for the lateness of this reply, I haven't been checking -bsd very regularly in recent days. It's a long email, which is common for me, but I do cover a lot of ground in it.) On Mon, Dec 01, 2003 at 10:56:30PM +0100, Robert Millan wrote: This is not about the Debian tools, you can have Debian tools even on NetBSD and get them into the ports collection. It's about the GNU libc and userland, which are the standard in Debian and I see no reason to replace them. There are other things in Debian that are also standard but that you do see a reason to replace, such as (most prominently) the kernel. Many valid technical reasons have been given as to why the NetBSD libc might well be more appropriate for a Debian port to the NetBSD kernel, but most of your replies have been boasts of the superiority of the GNU userland utilities and a desire to use the same libc as Linux and Hurd. I personally share your preference for the GNU utilities, but I quite strongly believe that it's a matter of personal preference, and I can definitely see how the NetBSD developers can reasonably disagree with me about that. In any case, everyone seems to be in agreement that the Debian port to the NetBSD kernel should use the GNU utilities, since a vast number of intentionally Debian-specific tools and scripts have been written to expect them. The same argument does not hold as well for the GNU libc, because the vast majority of the software in Debian is supposed to be portable enough to support NetBSD as an OS, and the NetBSD libc in particular. (If not, it's usually either a bug or else the author just didn't think about it one way or the other. In the latter case, most authors would be willing to accept a patch increasing portability.) Wishing to use the same libc as the other Debian ports is not obviously a bad argument, but it would need more justification. In fact, this superfically general rule is only one possible generalization of the fact that all current Debian ports use the GNU libc. Another possible generalization would be evident if you notice that the GNU libc is the primary, best integrated, and best supported libc on all existing Debian ports (including both Linux- and Hurd-based ones). This suggests that the rule should be to use the primary, best integrated, and best supported libc for every Debian port, which in the case of NetBSD would be the native NetBSD libc. Choosing between these generalizations requires more justification, and only the supporters of the latter generalization (the one that would chose NetBSD's native libc) have done so. They have done this justification by providing many technical examples of how their patches have been much less intrusive and how the porting process has gone much more smoothly with the native libc. In fact, there is reason to believe that this would be true for any other *NIX variant to which one were to make a Debian port (e.g., a hypothetical freed-as-in-speech AIX or Solaris). That successful thought experiment of extrapolation to potential future situations suggests that the second rule in the preceding paragraph generalizes better than the other one, and so it is the one to pick in absence of other conflicting information. That is why the native NetBSD libc seems to be the more productive way to go. If you still disagree with me, I encourage you to pursue your GNU libc-based version and present results when you have something concrete to share on this list for public testing. Until you have at least this to show, please don't belittle or trivialize the efforts of other people, who have spent a great deal of effort trying to implement the same functionality you are now trying to achieve, and who seem to have a great deal of experience with the area in general. I can't find the exact messages for some of these examples of their experience, but one post mentioned that the poster had implemented applications using hundreds or thousands of threads; Perry Metzger talked in another post about his and the NetBSD team's two-year-long ordeal to get scheduler activation working properly in the NetBSD kernel-and-libc package; and the author of another post had, apparently, individually implemented scheduler activation in a toy OS. These do not seem like people who are inexperienced in threads. I have less experience than they in this regard, which I freely admit, and which inclines me to believe them on this matter, especially since they are all in agreement. Finally, Matthew Garrett's post in which he criticized the style and tone of your emails, which you called a personal rant, seemed to me to have been written in a very professional and calm manner, with one minor exception (your complete inability to...). He did sound annoyed, but I and most probably numerous other readers of this list were also annoyed at your tone, for reasons that Matthew summarized well, and Matthew did seem to remain mostly polite while being annoyed Rather than not paying attention to
Re: Glibc-based Debian GNU/KNetBSD
Jimmy == Jimmy Kaplowitz [EMAIL PROTECTED] writes: Jimmy I can't find the exact messages for some of these examples of their Jimmy experience, but one post mentioned that the poster had implemented Jimmy applications using hundreds or thousands of threads; How can this be considered anything else than an evidence of mental illness ? (I purposedly avoid attributing it to malice). Or was this simply a pointless benchmark ? Jimmy Perry Metzger Jimmy talked in another post about his and the NetBSD team's two-year-long Jimmy ordeal to get scheduler activation working properly in the NetBSD Jimmy kernel-and-libc package; and the author of another post had, apparently, Jimmy individually implemented scheduler activation in a toy OS. These do not Jimmy seem like people who are inexperienced in threads. I have less Jimmy experience than they in this regard, which I freely admit, and which Jimmy inclines me to believe them on this matter, especially since they are Jimmy all in agreement. Well, if SA does not map with a minimum of a glue code to an 1:1 threading model, they've got not much to brag about. ~velco
Re: Glibc-based Debian GNU/KNetBSD
Jimmy == Jimmy Kaplowitz [EMAIL PROTECTED] writes: Jimmy On Fri, Dec 12, 2003 at 02:40:05PM +0200, Momchil Velikov wrote: Jimmy == Jimmy Kaplowitz [EMAIL PROTECTED] writes: Jimmy I can't find the exact messages for some of these examples of their Jimmy experience, but one post mentioned that the poster had implemented Jimmy applications using hundreds or thousands of threads; How can this be considered anything else than an evidence of mental illness ? (I purposedly avoid attributing it to malice). Or was this simply a pointless benchmark ? Jimmy What I meant by mentioning it was that this poster actually seemed to Jimmy have a legitimately useful (to him) application that legitimately needed Jimmy lots of threads, and getting it to work well on his development or test Jimmy system must have required a fair amount of familiarity with threads Jimmy and/or his OS's implementation of threads. And what I meant is that anyone writing an application with hundreds or thousands of threads should either choose another field or seek professional help. (Well, he/she might be the first one to make the breakthrough of finding such legitimately useful application, but I'd rather take the risk of wrongfully accusing him/her in incompetence). That's, of course, wrt. to the usefulness of the scheduler activations idea or, for that matter, of any N:M (N != 1 M != 1) threading architecture. Jimmy I believe the poster was Jimmy offering it to Robert as a way to test his eventual port of a threads Jimmy library to glibc-on-BSD to see if it performs well and is thread-safe Jimmy for thread-intensive applications such as his. (To give you an idea of Jimmy this poster's standards, he stated that he considered all versions of Jimmy Linux prior to the existence of NPTL not to be thread-safe for his Jimmy purposes.) What's the point in demonstrating how fast can you do nothing ? ~velco
Re: Glibc-based Debian GNU/KNetBSD
On Fri, Dec 12, 2003 at 02:40:05PM +0200, Momchil Velikov wrote: Jimmy == Jimmy Kaplowitz [EMAIL PROTECTED] writes: Jimmy I can't find the exact messages for some of these examples of their Jimmy experience, but one post mentioned that the poster had implemented Jimmy applications using hundreds or thousands of threads; How can this be considered anything else than an evidence of mental illness ? (I purposedly avoid attributing it to malice). Or was this simply a pointless benchmark ? What I meant by mentioning it was that this poster actually seemed to have a legitimately useful (to him) application that legitimately needed lots of threads, and getting it to work well on his development or test system must have required a fair amount of familiarity with threads and/or his OS's implementation of threads. I believe the poster was offering it to Robert as a way to test his eventual port of a threads library to glibc-on-BSD to see if it performs well and is thread-safe for thread-intensive applications such as his. (To give you an idea of this poster's standards, he stated that he considered all versions of Linux prior to the existence of NPTL not to be thread-safe for his purposes.) - Jimmy Kaplowitz [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: Glibc-based Debian GNU/KNetBSD
On Fri, Dec 12, 2003 at 04:41:39PM +0200, Momchil Velikov wrote: Jimmy I believe the poster was Jimmy offering it to Robert as a way to test his eventual port of a threads Jimmy library to glibc-on-BSD to see if it performs well and is thread-safe Jimmy for thread-intensive applications such as his. (To give you an idea of Jimmy this poster's standards, he stated that he considered all versions of Jimmy Linux prior to the existence of NPTL not to be thread-safe for his Jimmy purposes.) What's the point in demonstrating how fast can you do nothing ? Well, first of all, there probably are legitimate uses for it that neither of us has thought of, although I am willing to be told otherwise by someone who has looked into this. In that sense, it's probably not nothing to achieve fast and correct performance from the thread library in this regard. Yes, it has usefulness as an informal benchmark, or as part of a more rigorous formal benchmark. In any case, I hope I did indicate that I have less experience than many list posters with threads (although I hope to gain at least a bit more when I take an operating systems course at my uni as soon as next fall). If anything I said in the previous paragraph is rubbish, I'm quite willing to believe it. My point to Robert was primarily that most of the people making claims about threads were quite knowledgeable people, and if he was disagreeing with their easily-reached consensus, and voicing this disagreement on this list as forcefully as he was, he should have some concrete evidence (or else really good logic from premises that are agreed upon by everyone concerned) to back up his claims. If he doesn't do that, flamewars and hurt feelings ensue. Since I am not a threads expert, I personally tried not to make any claims about threads in my post to Robert beyond citing the statements of other posters who have demonstrated relevant knowledge or experience. - Jimmy Kaplowitz [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: Glibc-based Debian GNU/KNetBSD
On Fri, Dec 12, 2003 at 09:53:38AM -0500, Jimmy Kaplowitz wrote: In any case, I hope I did indicate that I have less experience than many list posters with threads (although I hope to gain at least a bit more when I take an operating systems course at my uni as soon as next fall). If anything I said in the previous paragraph is rubbish, I'm quite willing to believe it. For those without the benefit of a University course, I would suggest picking up one of the oldest classical texts on OS design principles and practice, though I suggest reading it with a critical eye: Operating Systems Design and Implementation, Second Edition Tannebaum, A. S. Woodhull, A. S. (Prentice Hall, 1997) Yes, this is the Tannebaum that Linus was arguing with on comp.os.minix, that brought about the creation of Linux. The book contains the entire source code for the Minix operating system, and a great deal of discussion on the whys and wherefores of the pieces, with references to the source. While primarily focused on message-passing kernel architectures, of course, it does provide a very strong background in all the fundamental pieces of a Unix-like operating system. It is, of course, far from the only option. :) -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU/KLNetBSD(i386) porter : :' : `. `' `- pgp0lgQsSrjMI.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
On Fri, Dec 12, 2003 at 08:54:01AM -0700, Joel Baker wrote: In any case, I hope I did indicate that I have less experience than many list posters with threads (although I hope to gain at least a bit more when I take an operating systems course at my uni as soon as next fall). If anything I said in the previous paragraph is rubbish, I'm quite willing to believe it. For those without the benefit of a University course, I would suggest picking up one of the oldest classical texts on OS design principles and practice, though I suggest reading it with a critical eye: Operating Systems Design and Implementation, Second Edition Tannebaum, A. S. Woodhull, A. S. (Prentice Hall, 1997) According to the course syllabus for this year's edition of the course, there is a draft of the course textbook available online, and they recommend but don't require another book by Tanenbaum (whose name they spell with no doubled N), which is _Modern Operating Systems_, Second Edition, by Andrew S. Tanenbaum, Prentice Hall, 2001. Some or all of the course textbook (written by the professor, no less) is online, but I'm not going to post a link on this mailing list because I don't know if the professor wants it to be generally available to the world at large. (If he doesn't want that, then he especially wouldn't want a link that's permanently and publically archived on many sites and quite findable via Google.) It's supposed to be a really intense course, wherein you write threads implementations, a simple VFS, a simple filesystem to work with the VFS, etc., and it's amazingly intense if you take the additional half-credit lab section where you write a large portion of a simplified *NIX-like OS called Weenix. ;-) Here's to hoping I have my study skills refined by then - Jimmy Kaplowitz [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: Glibc-based Debian GNU/KNetBSD
On Fri, Dec 12, 2003 at 11:19:39AM -0500, Jimmy Kaplowitz wrote: On Fri, Dec 12, 2003 at 08:54:01AM -0700, Joel Baker wrote: In any case, I hope I did indicate that I have less experience than many list posters with threads (although I hope to gain at least a bit more when I take an operating systems course at my uni as soon as next fall). If anything I said in the previous paragraph is rubbish, I'm quite willing to believe it. For those without the benefit of a University course, I would suggest picking up one of the oldest classical texts on OS design principles and practice, though I suggest reading it with a critical eye: Operating Systems Design and Implementation, Second Edition Tannebaum, A. S. Woodhull, A. S. (Prentice Hall, 1997) According to the course syllabus for this year's edition of the course, there is a draft of the course textbook available online, and they recommend but don't require another book by Tanenbaum (whose name they spell with no doubled N), which is _Modern Operating Systems_, Second Edition, by Andrew S. Tanenbaum, Prentice Hall, 2001. Some or all of the course textbook (written by the professor, no less) is online, but I'm not going to post a link on this mailing list because I don't know if the professor wants it to be generally available to the world at large. (If he doesn't want that, then he especially wouldn't want a link that's permanently and publically archived on many sites and quite findable via Google.) It's supposed to be a really intense course, wherein you write threads implementations, a simple VFS, a simple filesystem to work with the VFS, etc., and it's amazingly intense if you take the additional half-credit lab section where you write a large portion of a simplified *NIX-like OS called Weenix. ;-) Here's to hoping I have my study skills refined by then Given what I can see in various references, _Modern Operating Systems_ is basically the same as OSDI, but without the Minix source code (and, of course, references to it). However, the fact that it remains a primary coursebook today, and was a primary coursebook more than a decade ago, should say something about it's nature, given the changes in code that have occured since - to wit, that the fundamentals haven't much changed. -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU/KLNetBSD(i386) porter : :' : `. `' `- pgpnZAC6F3Sho.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 04:21:04PM -0500, Perry E.Metzger wrote: Joel Baker [EMAIL PROTECTED] writes: I haven't been following too closely. Could someone explain what the issue is? Obviously XFree works fine on NetBSD -- I'm using it at this very moment. Given that it works fine on NetBSD, what's the issue? Crucial issues mostly amounted to differences between libraries linked to on NetBSD and those linked to by default under Linux (which, prior to the patches, XFree86 assumed for Debian). [...] Apart from that, it was mostly stuff like where do man pages live, how do I call X userland utility, etc - most of that is in the NetBSD.cf patch. Okay, so it wasn't functional issues, it was integration issues. Gotcha. Just like with the Glibc-based port. I haven't had to write code for a new kernel, or for a new C library, since both KNetBSD (K for 'kernel of') and Glibc are supported in Xfree86. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 04:16:00PM -0500, Perry E.Metzger wrote: Why do you want NetBSD's kernel? You obviously believe everything that NetBSD has done is fecal, so what would the point of contaminating the superior GNU userland with a crappy NetBSD kernel? Well I don't recall using scathologic terms in this discussion. Asides from that, I find NetBSD's kernel quite good in comparison to Linux. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Wed, Dec 03, 2003 at 02:16:04AM +0100, [EMAIL PROTECTED] wrote: So your best bet to get your port released is to provide an environment as similar as the GNU/Linux so that most packages will build out of the box. Using glibc and GNU tools is a big step in this direction. Fully agreed. But sadly noone will believe that untill we add a substantial number of built packages for the Glibc-based port. Back to hacking (I hope) ;) -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 05:19:41PM +, Matthew Garrett wrote: On Tue, Dec 02, 2003 at 01:52:01AM +0100, Robert Millan wrote: Please avoid the third party euphemism. If you want to run non-free software on a Glibc-based system, you can use the NetBSD libc since it's no technical problem to provide it as alternative (ala Linux libc5) Third party includes the large body of Free software that isn't shipped by Debian, plus unmodified upstream code not supplied by Debian. There's no need whatsoever to start assuming that it's a euphemism. If read Perry's response, and me subsequent response, you'll find that Perry meant something completely different. You'll also find how I recognised my own confusion. Look, Robert, what is it about you and your complete inability to actually work with others whose goals aren't identically aligned to yours? Please send me your personal rants privately, or keep pestering me when I'm not present (like in #debian-devel). That way I don't have to pay any attention to you. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 08:45:52AM -0800, Michael Graff wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday 02 December 2003 07:58 am, Robert Millan wrote: No, we combine the advantages of Debian, GNU, and the kernel of NetBSD. The superiority of GNU userland repect to NetBSD's is an issue too, and you seem to be ignoring it. I'm sorry, but this I have to respond to. No reason to be sorry for that. I don't see the GNU userland as supperior. I see it as annoying. GNU isn't better, just different. And those differences drive me nuts. Well I guess I should learn to keep statements I'm not willing to defend for myself. I'm not really interested on discussing this. But, that asside... what you have been proposing seems to be taking NetBSD and removing all but the kernel. In that case, it's not NetBSD anymore. You're right. The system is basicaly a GNU variant with another kernel. That aside, insulting a group you're trying to work with seems counter-productive. I didn't mean to insult. That was just my perspective. Are you certain you're not Stallman in disguise? Heh. You caught me ;). If you insist on that I'll have to start signing my mail *g*. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 08:16:35PM -0700, Joel Baker wrote: The NetBSD/native port has been stalled for some time, because I ran into core, required-to-build-lots-of-things applications (tcl8.4, IIRC, in particular) that *don't work* with GNU pth. Period. Neither version 1.x or 2.x. I'm not sure if you meant something else but, from pristine sources: $ ldd /usr/lib/libtcl8.4.so.0 libc.1 = /lib/libc.so.1 (0x) libdl.2 = /lib/libdl.so.2 (0x) libpthread.20 = /usr/lib/libpthread.so.20 (0x) libm.1 = /lib/libm.so.1 (0x) (the 0x0 is due to a hack in our ldd wrapper) -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Thu, Dec 04, 2003 at 03:50:00PM +0100, Robert Millan wrote: On Mon, Dec 01, 2003 at 08:16:35PM -0700, Joel Baker wrote: The NetBSD/native port has been stalled for some time, because I ran into core, required-to-build-lots-of-things applications (tcl8.4, IIRC, in particular) that *don't work* with GNU pth. Period. Neither version 1.x or 2.x. I'm not sure if you meant something else but, from pristine sources: $ ldd /usr/lib/libtcl8.4.so.0 libc.1 = /lib/libc.so.1 (0x) libdl.2 = /lib/libdl.so.2 (0x) libpthread.20 = /usr/lib/libpthread.so.20 (0x) libm.1 = /lib/libm.so.1 (0x) (the 0x0 is due to a hack in our ldd wrapper) Don't work, not don't link... though it could well be a problem where GNU pth is horking over something involving the libc, which is (of course) different as well. -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpLTI5F5sWfE.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
On Mon, 1 Dec 2003, Perry E.Metzger wrote: party. Our pkgsrc infrastructure exists to make it easy to compile third party software, but we do not claim that Emacs and /bin/ls are supported the same way. We've got about 4500 packages in pkgsrc -- a fraction of the number some folks like Debian support, but quite a number -- and in the Counting packages doesn't work well. Debian splits up many, many software suites into separate packages, such as docs, libraries, programs/ executables, development headers, and shared data. It is common to have one package in pkgsrc be represented by three-or-more Debian packages. Only for a few packages in pkgsrc are they split up. (I do agree that separating big software into seperate packages is a good idea -- I use new freedesktop.org xlibs which is in over 15 packages.) course of making them all work we routinely find that we have to fix things in NetBSD. For example, programs like xmms have inspired many changes to our threads system. Another thing that is interesting is that most of pkgsrc is usable on non-NetBSD systems. Many admins use it to have a consistent third-party software installation method under Solaris and Linux. pkgsrc is used (or has been used) under BSD/OS, Mac OS X, Darwin OS, Irix, FreeBSD, OpenBSD, HP-UX, and others. In fact, one pkgsrc developer is beginning to use pkgsrc (via mingw compiler) to provide software for Windows platforms. I even use pkgsrc to build and install Linux the kernel, glibc, iptables, vixie-cron, shadow, all components of my Linux distribution. (A few of these pkgsrc packages are not in pkgsrc or pkgsrc-wip yet though.) Jeremy C. Reed echo '9,J8HD,[EMAIL PROTECTED]:[EMAIL PROTECTED];[EMAIL PROTECTED]@5GBIELD54DL@8L?:5GDEJ8LDG1' |\ sed ss,s50EBsg | tr 0-M 'p.wBt SgiIlxmLhan:o,erDsduv/cyP'
Re: Glibc-based Debian GNU/KNetBSD
Perry == Perry E Metzger [EMAIL PROTECTED] writes: Perry Why do you want NetBSD's kernel? You obviously believe everything that Perry NetBSD has done is fecal, so what would the point of contaminating the Perry superior GNU userland with a crappy NetBSD kernel? Done with the strawman already ?
re: Glibc-based Debian GNU/KNetBSD
This may be possible under NetBSD as well (especially with a generous helping of COMPAT option enabling), but given the number of dire warnings the manuals all bear about building things in the correct order, I'm not willing to trust that it's flexible enough to start doing interesting things with. FWIW, generally it's considered a major bug if a new netbsd kernel[*] fails with an old userland. (_maybe_ ld.elf_so needs to be updated in some rare cases that only apply to some platforms once.) most of the warnings are (somewhat) dated with the new ./build.sh method of building netbsd. in general the only rule one has to adhere to is userland age = kernel age. and don't build to DESTDIR=/ :-) but even that's not always the case - a 1.5 kernel can run nearly all of the 1.6 userland (this isn't going to be true of 1.6 and 2.0, nor is it true in most other major releases.) .mrg. [*] this is with all the the COMPAT_* options you meantion enabled.
Re: Glibc-based Debian GNU/KNetBSD
On Wed, Dec 03, 2003 at 09:30:09AM -0800, Jeremy C. Reed wrote: On Wed, 3 Dec 2003, Joel Baker wrote: Another thing that is interesting is that most of pkgsrc is usable on non-NetBSD systems. Many admins use it to have a consistent third-party software installation method under Solaris and Linux. pkgsrc is used (or has been used) under BSD/OS, Mac OS X, Darwin OS, Irix, FreeBSD, OpenBSD, HP-UX, and others. In fact, one pkgsrc developer is beginning to use pkgsrc (via mingw compiler) to provide software for Windows platforms. I even use pkgsrc to build and install Linux the kernel, glibc, iptables, vixie-cron, shadow, all components of my Linux distribution. (A few of these pkgsrc packages are not in pkgsrc or pkgsrc-wip yet though.) Of course, the same is true of dpkg/apt; with fairly minor changes, dpkg / apt can be built on nearly any modern POSIX system (in fact, the only issue with NetBSD was a non-POSIX-compliance issue). Which is to say, they're both portable, useable management tools, and so it isn't suprising that people port them and use them to manage things. :) Yes, I understand that. The point is that pkgsrc is already used on various platforms and most of the build instructions/patching has been done to configure, build, and install correctly on a wide variety of platforms. I'd be curious to know if dpkg/apt is regularily used on Solaris, for Windows, HPUX, IRIX, BSD/OS, normal FreeBSD, normal OpenBSD. (If I recall correctly, I think soem derivative is used for some Darwin OS.) I know of at least one effort to get it on Solaris; I don't keep track of the rest enough to know for sure (though I know FreeBSD, like NetBSD, is compatible for the simple fact that we need it to be). Also, many of the *diff.gz patches for the dpkg sources are not portable across Irix, Solaris, NetBSD, etc. (I often review debian source diff.gz files to get ideas for patches and to see build and configuration methods.) Er. dpkg is a native Debian tool; it has no diff.gz file. So I'm not entirely sure what you mean here, unless you mean debian patches against origional sources to build Debian packages - any patches which make such things incompatible with the NetBSD core, at least, will end up under quite a bit of scrutiny. For obvious reasons. By the way, I like Debian packages too! apt is very useful :) And if pkgsrc were the only thing I wanted to replace on NetBSD, well, I'd just replace it with dpkg and apt. But since my interest is largely in the administrative consistancy of Debian, and the policy rules that it enforces on it's packages, among other things... -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpXvYgPE0dvo.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
Jeremy C. Reed [EMAIL PROTECTED] writes: On Mon, 1 Dec 2003, Perry E.Metzger wrote: party. Our pkgsrc infrastructure exists to make it easy to compile third party software, but we do not claim that Emacs and /bin/ls are supported the same way. We've got about 4500 packages in pkgsrc -- a fraction of the number some folks like Debian support, but quite a number -- and in the Counting packages doesn't work well. Debian splits up many, many software I was not counting packages. I was merely saying that we have fixed our libc so that it is highly compatible, and that we continue to strive to do better. -- Perry E. Metzger[EMAIL PROTECTED]
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 09:26:33PM -0500, Perry E.Metzger wrote: Robert Millan [EMAIL PROTECTED] writes: It isn't the FSF who maintains Glibc. Whatever. The point is the maintainers won't do the work for you, so the upstream comment doesn't hold much water. The primary target of Glibc is Hurd/Mach, but still Linux is much better supported. This is because GNU/Linux is a mature system while GNU/Hurd is not yet. The maintainers will support a particular system if they're interested in it, and I have reasons to believe some of them will be interested in GNU/K*BSD. That's not what third party means. Third party means stuff not provided by The NetBSD Foundation in our releases. The BSD world doesn't work quite the way the Linux world does in this regard. We maintain both a kernel and a tightly integrated userland that goes with it -- anything else, for example, Gnu Emacs, is third party. Our pkgsrc infrastructure exists to make it easy to compile third party software, but we do not claim that Emacs and /bin/ls are supported the same way. This is Debian, and we have around 1 packages here. Why do we have to support installing packages from the NetBSD pkgsrc archive? The threads stuff is not as relevant nor as complicated as you pretend. I've been working with libpth pth is not a real preemptive threads library. It doesn't work very well as a result. You need kernel support to do threads right, and the kernel support is necessarily intimately connected to the userland implementation that interfaces with it. I know that. The same applies to the rest of libc. At some point we'll study which of porting NPTL or porting the pthreads from NetBSD's libc is a more viable option. You will never port NPTL, period, because to do that would require that you change the NetBSD kernel, and you aren't in a position to do that. Porting the pthreads from NetBSD's libc will be a very serious job, and as I noted will require tracking all the changes that get made in NetBSD's pthreads engine on a constant basis, because as I said, there is a protocol that the kernel and userland parts of the pthreads implementation have to agree on. We can do either. If it's a serious job, that's fine. I'm not really worried about that. If you want to help or propose a solution, that's fine, but stop playing devil's advocate which is just a waste of my time. I'm not playing devil's advocate. I'm trying to tell you the truth to try to help you. You seem to be unwilling to listen. Thus, I'll drop out of the conversation. Do whatever you want. As a member of the NetBSD project, I have my doubts that your main interest here is helping me. We have had this discussion before. I explained a hundred times why porting the bulk of Debian is much easier with Glibc, but again some people are unwilling to listen. They have their reasons, of course, but this is why I'm proving it with facts, so I don't have to engage in eternal discussions. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 10:12:08PM -0500, Nathan Hawkins wrote: On Tue, Dec 02, 2003 at 01:52:01AM +0100, Robert Millan wrote: Please avoid the third party euphemism. If you want to run non-free software on a Glibc-based system, you can use the NetBSD libc since it's no technical problem to provide it as alternative (ala Linux libc5) He's referring to source compatibility. On BSD's, third party seems to mostly mean just about anything not in the base system. NetBSD has binary emulation for Linux. That takes care of the non-free stuff, since that's far more available on Linux than for NetBSD. We have source compatibility with all other Glibc-based GNU variants, which includes such a popular system as GNU/Linux. I think it's much more compatibility than what the NetBSD libc can provide. No, _you_ are underestimating the complexities. There is a very large difference between something like libpth or linuxthreads (which are either entirely based in userspace, or require minimal support from the kernel), and the very kernel-specific implementations being used in NPTL, FreeBSD's libkse or NetBSD's libpthreads. The newer threads packages are targeting drastic improvements in performance, Posix compliance, stability, and scalability. And from what I understand, they're getting them. [...] Thanks for clarifiing. Well I don't see the point of discussing all this at this early point, but since all of you are insisting.. In short term, we can use non-preemptive threads (or linuxthreads) without much problem. If there are API incompatibilities with pthreads standard (which there shouldn't be), we fix them. In long term, we can either: - Port NPTL (this makes sense since NPTL is the library Hurd developers are targetting, for example). - Port libkse and libpthreads. We'd have to integrate it with Glibc interfaces, and reformat API to be POSIX compliant if applicable. And don't tell me the long term solutions are too much work. They're long term for some reason. The system is pretty usable with non-preemptive thread emulation. What is more relevant from the portability perspective is to provide POSIX threads. We can provide POSIX threads with libpth but AFAICS libkse/pthreads are not POSIX threads compliant. If that is the case, note that it is much more detrimental for the port to provide non-standard threads with high performance than to provide POSIX threads even if they use a 100% userspace non-preemptive emulation (libpth). -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
Nathan Hawkins [EMAIL PROTECTED] writes: (btw, fixing the X server is on my todo) All I have to say about the X server, as the person who generated most of the patches, is that they're actually very straightforward, if rather invasive. I simply had to go through each config option and decide whether it should be handled in the 'native' way, or the GNU-userland way (and it was very much a userland issue, not a libc issue). The *hard* part was in hunting down build problems and bad assumptions in something the size of the X codebase. That isn't going to be any saner on a Glibc+FreeBSD system; probably less sane, in fact. That was pretty much my experience, too. In fact, I tried to get it working on glibc, and had fits with it. I particularly remember xterm being a disaster. I gave up on it, and Robert evidently got it working except for the server. It probably needs some headers in sys/ that glibc didn't get right. I haven't been following too closely. Could someone explain what the issue is? Obviously XFree works fine on NetBSD -- I'm using it at this very moment. Given that it works fine on NetBSD, what's the issue? Perry
Re: Glibc-based Debian GNU/KNetBSD
Robert Millan [EMAIL PROTECTED] writes: That's not what third party means. Third party means stuff not provided by The NetBSD Foundation in our releases. The BSD world doesn't work quite the way the Linux world does in this regard. We maintain both a kernel and a tightly integrated userland that goes with it -- anything else, for example, Gnu Emacs, is third party. Our pkgsrc infrastructure exists to make it easy to compile third party software, but we do not claim that Emacs and /bin/ls are supported the same way. This is Debian, and we have around 1 packages here. Why do we have to support installing packages from the NetBSD pkgsrc archive? Are you deliberately misreading me? I wasn't saying anything about you supporting pkgsrc. I was explaining what third party means in this context, and why it is that we try to make our libc support everything we can. The threads stuff is not as relevant nor as complicated as you pretend. I've been working with libpth pth is not a real preemptive threads library. It doesn't work very well as a result. You need kernel support to do threads right, and the kernel support is necessarily intimately connected to the userland implementation that interfaces with it. I know that. Then why are you comparing pth to the job of making a kernel assisted pthreads system work? They are unrelated. At some point we'll study which of porting NPTL or porting the pthreads from NetBSD's libc is a more viable option. You will never port NPTL, period, because to do that would require that you change the NetBSD kernel, and you aren't in a position to do that. Porting the pthreads from NetBSD's libc will be a very serious job, and as I noted will require tracking all the changes that get made in NetBSD's pthreads engine on a constant basis, because as I said, there is a protocol that the kernel and userland parts of the pthreads implementation have to agree on. We can do either.[...] I give up. If you think you can make NPTL somehow work on an operating system that doesn't have the kernel support for it, well, good luck to you. For the record, if anyone OTHER than Robert is interested in discussing various things that NetBSD could do to make it easier for people to use our stuff with Debian, please get in touch with me. We're always interested in making improvements to our available APIs etc. Perry
Re: Glibc-based Debian GNU/KNetBSD
Joel Baker [EMAIL PROTECTED] writes: While I'd dearly love to see a bit more de-coupling of NetBSD kernel and libc (so that they don't have to be in quite such lockstep, though I'm more worried about the process utilities that must be *exact* matches), I don't claim that managing it would be trivial, and may not even be practical (or at least, might not be feasible short of 3.0). BTW, NetBSD gets a lot out of the tight integration of the libraries and kernel. What we get is completely seamless integration -- and that buys us quite a bit. In general, I can see why one would want to combine the advantages of Debian and NetBSD, but I can't see why one would want to produce something less functional than either. I think if you replace too much of NetBSD in doing what you're trying to do, you just end up with a crippled Linux rather than something synergetic. Perry
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 10:22:53PM -0500, Nathan Hawkins wrote: All I have to say about the X server, as the person who generated most of the patches, is that they're actually very straightforward, if rather invasive. I simply had to go through each config option and decide whether it should be handled in the 'native' way, or the GNU-userland way (and it was very much a userland issue, not a libc issue). The *hard* part was in hunting down build problems and bad assumptions in something the size of the X codebase. That isn't going to be any saner on a Glibc+FreeBSD system; probably less sane, in fact. That was pretty much my experience, too. In fact, I tried to get it working on glibc, and had fits with it. I particularly remember xterm being a disaster. I gave up on it, and Robert evidently got it working except for the server. It probably needs some headers in sys/ that glibc didn't get right. Porting Xfree86 with Glibc was just a task of fixing minor build errors. Look at my patches, most of them are one-liners. As for xterm, there were only two hunks iirc. One to s/__GNU__/__GLIBC__/g and another to include termios.h. Just look at the comparison (in my local tree): $ wc -l debian/patches/84* debian/patches/000_stolen_from_HEAD_netbsd.diff 51 debian/patches/840_netbsd_bsdLib.rules_fix.diff 36 debian/patches/841_netbsd_imake.c_fixes.diff 190 debian/patches/842_netbsd_NetBSD.cf_fixes.diff 26 debian/patches/843_netbsd_no_shared_OldX_lib.diff 19 debian/patches/844_netbsd_no_kbd_mode_command.diff 195 debian/patches/000_stolen_from_HEAD_netbsd.diff 517 total $ wc -l debian/patches/82* 100 debian/patches/820_gnu-kbsd_config.diff 16 debian/patches/822_gnu-kbsd_xload.diff 37 debian/patches/823_gnu-kbsd_xterm.diff 153 total And note the name gnu-kbsd, not gnu-kfreebsd. None of these patches is KFreeBSD-specific. 820, 822 and 823 actualy fix it for all new Glibc-based GNU variants (even non-K*BSD ones). -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 09:41:48AM -0500, Perry E.Metzger wrote: I haven't been following too closely. Could someone explain what the issue is? Obviously XFree works fine on NetBSD -- I'm using it at this very moment. Given that it works fine on NetBSD, what's the issue? Joel will give you the details. But the issue is, in short: Debian is _not_ NetBSD. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 09:52:43AM -0500, Perry E.Metzger wrote: This is Debian, and we have around 1 packages here. Why do we have to support installing packages from the NetBSD pkgsrc archive? Are you deliberately misreading me? I wasn't saying anything about you supporting pkgsrc. I was explaining what third party means in this context, and why it is that we try to make our libc support everything we can. Ah, excuse me I didn't fully understand you. Well, the number of supported apps with Glibc is much bigger than the one with NetBSD libc. pth is not a real preemptive threads library. It doesn't work very well as a result. You need kernel support to do threads right, and the kernel support is necessarily intimately connected to the userland implementation that interfaces with it. I know that. Then why are you comparing pth to the job of making a kernel assisted pthreads system work? They are unrelated. They can be API compatible if you follow POSIX, which is enough for now. For the record, if anyone OTHER than Robert is interested in discussing various things that NetBSD could do to make it easier for people to use our stuff with Debian, please get in touch with me. We're always interested in making improvements to our available APIs etc. Since we still use the kernel of NetBSD, there are things you could do to help in that regard. Do you want me to send you the details from now on? -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 10:03:11AM -0500, Perry E.Metzger wrote: BTW, NetBSD gets a lot out of the tight integration of the libraries and kernel. What we get is completely seamless integration -- and that buys us quite a bit. In general, I can see why one would want to combine the advantages of Debian and NetBSD, but I can't see why one would want to produce something less functional than either. I think if you replace too much of NetBSD in doing what you're trying to do, you just end up with a crippled Linux rather than something synergetic. No, we combine the advantages of Debian, GNU, and the kernel of NetBSD. The superiority of GNU userland repect to NetBSD's is an issue too, and you seem to be ignoring it. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 07:55:39PM -0700, Joel Baker wrote: On Mon, Dec 01, 2003 at 10:56:30PM +0100, Robert Millan wrote: It's about the GNU libc and userland, which are the standard in Debian and I see no reason to replace them. For the record, probably 70% of the email that I've seen on this list falls somewhere under the category of a single, perpetual, floating discussion/argument/flamewar. To wit: NetBSD/FreeBSD/OpenBSD? GNU libc or native libc? (I've never seen another alternative, IIRC) GNU userland or native userland? I agree with you. This discussion is just the nth iteration of the same waste of time. I'll go back to hacking now.. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday 02 December 2003 07:58 am, Robert Millan wrote: No, we combine the advantages of Debian, GNU, and the kernel of NetBSD. The superiority of GNU userland repect to NetBSD's is an issue too, and you seem to be ignoring it. I'm sorry, but this I have to respond to. I don't see the GNU userland as supperior. I see it as annoying. GNU isn't better, just different. And those differences drive me nuts. But, that asside... what you have been proposing seems to be taking NetBSD and removing all but the kernel. In that case, it's not NetBSD anymore. NetBSD works as well as it does because libc and all the other system components are integrated and work well together. You'll never end up with a libc vs. kernel issue, where I've seen many people hurt themselves by upgrading package X which upgraded glibc on linux which then broke half the other binaries. That aside, insulting a group you're trying to work with seems counter-productive. Are you certain you're not Stallman in disguise? - --Michael -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (NetBSD) iD8DBQE/zMHAl6Nz7kJWYWYRAnswAJ9xN3r18Qbp/ZLoPO+7VKv2CDY1HQCfbKon lRA6T1clS3BkMMvNrPBO0f0= =avHU -END PGP SIGNATURE-
Re: Glibc-based Debian GNU/KNetBSD
Perry == Perry E Metzger [EMAIL PROTECTED] writes: Perry In general, I can see why one would want to combine the advantages of Perry Debian and NetBSD, but I can't see why one would want to produce Perry something less functional than either. Less functional is apparently not the goal, no matter is can indeed be used to describe an intermediate state. Perry I think if you replace too much Perry of NetBSD in doing what you're trying to do, you just end up with a Perry crippled Linux rather than something synergetic. I fail to see how combining NetBSD kernel with the GNU system would produce the Linux kernel. ~velco
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 09:41:48AM -0500, Perry E. Metzger wrote: Nathan Hawkins [EMAIL PROTECTED] writes: (btw, fixing the X server is on my todo) All I have to say about the X server, as the person who generated most of the patches, is that they're actually very straightforward, if rather invasive. I simply had to go through each config option and decide whether it should be handled in the 'native' way, or the GNU-userland way (and it was very much a userland issue, not a libc issue). The *hard* part was in hunting down build problems and bad assumptions in something the size of the X codebase. That isn't going to be any saner on a Glibc+FreeBSD system; probably less sane, in fact. That was pretty much my experience, too. In fact, I tried to get it working on glibc, and had fits with it. I particularly remember xterm being a disaster. I gave up on it, and Robert evidently got it working except for the server. It probably needs some headers in sys/ that glibc didn't get right. I haven't been following too closely. Could someone explain what the issue is? Obviously XFree works fine on NetBSD -- I'm using it at this very moment. Given that it works fine on NetBSD, what's the issue? Crucial issues mostly amounted to differences between libraries linked to on NetBSD and those linked to by default under Linux (which, prior to the patches, XFree86 assumed for Debian). That's why the Imake patches exist (to provide some concept of distinction). Apart from that, it was mostly stuff like where do man pages live, how do I call X userland utility, etc - most of that is in the NetBSD.cf patch. Frankly, to give youa *complete* answer, I'd have to go read the patches again, myself; it's been almost a year since I worked on them actively, and I long ago dumped state (and yes, I need to redo a bunch of it, potentially, for 4.3/4.4; that's on my list after getting -current/pre-2.0 working and buildable, and ensuring GCC works with it). -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpMLV65fDZnc.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 10:03:11AM -0500, Perry E. Metzger wrote: Joel Baker [EMAIL PROTECTED] writes: While I'd dearly love to see a bit more de-coupling of NetBSD kernel and libc (so that they don't have to be in quite such lockstep, though I'm more worried about the process utilities that must be *exact* matches), I don't claim that managing it would be trivial, and may not even be practical (or at least, might not be feasible short of 3.0). BTW, NetBSD gets a lot out of the tight integration of the libraries and kernel. What we get is completely seamless integration -- and that buys us quite a bit. In general, I can see why one would want to combine the advantages of Debian and NetBSD, but I can't see why one would want to produce something less functional than either. I think if you replace too much of NetBSD in doing what you're trying to do, you just end up with a crippled Linux rather than something synergetic. A clarification: when I say 'decouple', I don't really mean make it possible to be an arbitrary libc on top; I mean, more, decouple the very tight versioning limitations - one of the few places that Glibc+Linux beats NetBSD's kernel+libc is that Glibc can run on 2.2, 2.4, 2.6, probably 2.0 (thoguh I think modern Glibc has issues with that, anymore) - all with just a reboot. Even when the Linux kernel is newer than the Glibc, it copes fairly well; most of what's missing is new, better-optomized functionality from the kernel that the libc can't take advantage of until it's upgraded. This may be possible under NetBSD as well (especially with a generous helping of COMPAT option enabling), but given the number of dire warnings the manuals all bear about building things in the correct order, I'm not willing to trust that it's flexible enough to start doing interesting things with. Granted, this is much less of an issue for this port, compared to Linux, since it's fairly straightforward to figure out what kernel version things have to be build against; it's just inconvenient for users building their own kernels from other sources, at times. -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpuKXhyjzodl.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
Robert Millan [EMAIL PROTECTED] writes: In general, I can see why one would want to combine the advantages of Debian and NetBSD, but I can't see why one would want to produce something less functional than either. I think if you replace too much of NetBSD in doing what you're trying to do, you just end up with a crippled Linux rather than something synergetic. No, we combine the advantages of Debian, GNU, and the kernel of NetBSD. The superiority of GNU userland repect to NetBSD's is an issue too, and you seem to be ignoring it. Why do you want NetBSD's kernel? You obviously believe everything that NetBSD has done is fecal, so what would the point of contaminating the superior GNU userland with a crappy NetBSD kernel? -- Perry E. Metzger[EMAIL PROTECTED]
Re: Glibc-based Debian GNU/KNetBSD
Joel Baker [EMAIL PROTECTED] writes: I haven't been following too closely. Could someone explain what the issue is? Obviously XFree works fine on NetBSD -- I'm using it at this very moment. Given that it works fine on NetBSD, what's the issue? Crucial issues mostly amounted to differences between libraries linked to on NetBSD and those linked to by default under Linux (which, prior to the patches, XFree86 assumed for Debian). [...] Apart from that, it was mostly stuff like where do man pages live, how do I call X userland utility, etc - most of that is in the NetBSD.cf patch. Okay, so it wasn't functional issues, it was integration issues. Gotcha. -- Perry E. Metzger[EMAIL PROTECTED]
re: Glibc-based Debian GNU/KNetBSD
No, we combine the advantages of Debian, GNU, and the kernel of NetBSD. The superiority of GNU userland repect to NetBSD's is an issue too, and you seem to be ignoring it. no, it's more that we (at least perry and i, and most of the netbsd developers) _don't think GNU userland is superior_. but that's not the point. the point is that debian (bsd) is about providing the choice... it doesn't matter what any one thinks is superior.. .mrg.
Re: Glibc-based Debian GNU/KNetBSD
Dear debian-bsd folk, [disclaimer: I am just a random Debian developer, I don't use nor plan to use FreeBSD or NetBSD] For Debian to target for release a new port, the port has to match the release minima which are: 1) all base package 2) 90% of all packages build 3) a working installer 4) suitable machine to handle security updates. Before the port is targeted for release, Debian developers are not required to make any effort toward making their packages build on the port, even applying patches. Debian developers are mostly GNU/Linux users and are likely to use GNU specific features, and not ready to stop this usage for a port that have yet to happen. 90% of all packages represent curently 6840 (source) packages, but is likely to double every 2 years. So your best bet to get your port released is to provide an environment as similar as the GNU/Linux so that most packages will build out of the box. Using glibc and GNU tools is a big step in this direction. Coming with a distribution with less feature/efficiency than the original *BSD flavour is not a problem as long as said feature are not part of the release criteria. Alternatively, maybe you don't want to release as a Debian port, and you prefer with a well-working set of packages with custom patches. In this case you have much more leeway, but then I question whether this should be endorsed by Debian at all, though you are welcome to do it. Cheers, -- Bill. [EMAIL PROTECTED] Imagine a large red swirl here.
Re: Glibc-based Debian GNU/KNetBSD
On Wed, Dec 03, 2003 at 02:16:04AM +0100, [EMAIL PROTECTED] wrote: Dear debian-bsd folk, [disclaimer: I am just a random Debian developer, I don't use nor plan to use FreeBSD or NetBSD] For Debian to target for release a new port, the port has to match the release minima which are: 1) all base package 2) 90% of all packages build 3) a working installer 4) suitable machine to handle security updates. Before the port is targeted for release, Debian developers are not required to make any effort toward making their packages build on the port, even applying patches. Debian developers are mostly GNU/Linux users and are likely to use GNU specific features, and not ready to stop this usage for a port that have yet to happen. 90% of all packages represent curently 6840 (source) packages, but is likely to double every 2 years. So your best bet to get your port released is to provide an environment as similar as the GNU/Linux so that most packages will build out of the box. Using glibc and GNU tools is a big step in this direction. Coming with a distribution with less feature/efficiency than the original *BSD flavour is not a problem as long as said feature are not part of the release criteria. Alternatively, maybe you don't want to release as a Debian port, and you prefer with a well-working set of packages with custom patches. In this case you have much more leeway, but then I question whether this should be endorsed by Debian at all, though you are welcome to do it. Alternatively to that, we can simply continue to submit patches that make the code more robust, and have a very high adoption rate upstream, since they almost invariably fix coding errors or problems that cause portability to be lost. Probably 80% of the packages I've touched that are *not* toolchain/core packages in some fashion, or very intricate, work straight out of the box. No changes whatsoever. Granted, getting 'base' to 100% is a much more painful prospect, but since GCC already has the work done, XFree86 has the work done barring maintenence on new X releases (IE, make sure it still works), Perl has worked since day 3 (at least for me)... really, a large part of the nasty stuff is past. Let me put it this way: if something *depends* on things provided by Glibc, it is so non-portable that I would seriously question whether it is, in fact, useful at all. The primary exemption that I can think of (psutils and kin) are already on my list of things to make sure we have replacements for that are suitable (since they are *very* tightly bound to the kernel, on NetBSD, I'm strongly considering having a 'kernutils' package which is generated along with kernel-image, kernel-source, etc). That, or convince NetBSD to provide enough support at the libc level so that the GNU process utilities can actually run linked against it, but since that seems like a lot more work for marginal reward, I haven't really investigated just how deep that would have to go. Might be useful, if someone is looking for a short-term project, for them to research that. (hint, hint :) Of the release criteria, I would say: 1) I used to have a webpage up; something like 70% of base was working in a reasonable fashion, if not all perfectly, and the rest looked like cleanup and half a dozen major issues (where I define 'major' as requiring most of my free cycles for a week or two to resolve meaningfully). 2) Is a matter of having a buildd (what some of my new hardware is intended to serve as, at least for the time being and possbily in an ongoing fashion), and spending the time to submit patches to fix what doesn't work. As I said above, probably 80% of the non-massive packages I've dealt with came out without significant problems. Which puts us well along the path towards a 90% build rate, right there, just by having a buildd. 3) I've talked to the debian-installer folks about this, and we've hashed over parts extensively. Right now, it's been less of a priority for me, personally, because I run everything in a chroot from a native box, but it will need to happen eventually. And will probably involve a few major changes to d-i to support the BSDs, but probably only *one* set of them for the entire set of BSD ports (whichever ones make it eventually to useful levels of completeness). 4) This, I can't really speak to, except to say that I'm already donating one machine (and network connection) to the cause, full-time, on my own dime. Perhaps it's something we can use one of those ever-mentioned spare i386 boxes for. Realistically, this is one of the *last* issues, since a combination of 2 and 3 leads to the obvious result of we can install a machine, and we can run a buildd on a machine - makes it fairly straightforward to set up things and give them to the security team, once we have the hardware. And though you didn't mention it, in my mind at least, there's: 5) The 'look and feel' should not be jarringly different from any
Re: Glibc-based Debian GNU/KNetBSD
On Wed, Dec 03, 2003 at 02:16:04AM +0100, [EMAIL PROTECTED] wrote: Debian developers are mostly GNU/Linux users and are likely to use GNU specific features, and not ready to stop this usage for a port that have yet to happen. The vast majority of code shipped by Debian is not Debian-specific and should not be Linux specific. In those cases, failing to build on NetBSD can be reasonably considered a bug (in most cases, there's no good reason to use glibc or Linux specific code) or suggest a useful feature that's missing in the NetBSD libc. I've met little resistance from people regarding the first of these, and the NetBSD folks have been extremely happy to add things that fall into the latter. So your best bet to get your port released is to provide an environment as similar as the GNU/Linux so that most packages will build out of the box. Using glibc and GNU tools is a big step in this direction. I think using GNU tools is certainly likely to happen - I think it would be hard to reasonably argue that it's a Debian system unless ls --color works... Glibc is, I think, less important. From the packages I've tried, very few failures have been due to glibc/libc issues - frankly, I've had more trouble with packages that notice that they're building on NetBSD and so want to produce manpages in cat format, which is something that's true of any port. Coming with a distribution with less feature/efficiency than the original *BSD flavour is not a problem as long as said feature are not part of the release criteria. Uh, of course it is. Producing software that sucks but happens to satisfy release criteria does little to benefit Debian, our users or the free software community. Doing it right is more important, even if that means it takes longer. -- Matthew Garrett | [EMAIL PROTECTED]
Re: Glibc-based Debian GNU/KNetBSD
On Sat, Nov 29, 2003 at 03:45:15PM +0100, Pavel Cahyna wrote: Hello, Hi! A usable Debian GNU/KNetBSD system based on Glibc is now available: http://people.debian.org/~rmh/knetbsd/pub/ I got to port all the base system with very little effort, since my previous patches for Glibc-based GNU/KFreeBSD already did most of the job. Xfree86 and Please, how is threading implemented in the NetBSD (and FreeBSD) port of Glibc? The goal is porting NPTL for both KFreeBSD and KNetBSD (K for 'kernel of'). As of now, temporary solutions are being used: linuxthreads and libpth, respectively. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Sun, Nov 30, 2003 at 09:46:19PM -0500, Perry E.Metzger wrote: IMHO, the amount of work involved in making glibc stably work with scheduler activations is likely prohibitive. You'll be chasing problems in the library forever. First we'll merge the patchset in upstream. Then we'll have problems for a while, similarly to those the GNU/Hurd port has fixing Glibc every time it breaks for them. But unlike GNU/Hurd, at some point upstream developers will install GNU/K*BSD themselves and maintain it for us. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 01:34:42PM +0100, Robert Millan wrote: On Sun, Nov 30, 2003 at 09:46:19PM -0500, Perry E.Metzger wrote: IMHO, the amount of work involved in making glibc stably work with scheduler activations is likely prohibitive. You'll be chasing problems in the library forever. First we'll merge the patchset in upstream. Then we'll have problems for a while, similarly to those the GNU/Hurd port has fixing Glibc every time it breaks for them. I'd be more impressed with that analysis if the bugs were in fact getting fixed. So far, DNS is _still_ broken, after more than a year. But unlike GNU/Hurd, at some point upstream developers will install GNU/K*BSD themselves and maintain it for us. How do you know that? Why should they do this when they didn't for the Hurd? ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
Robert Millan [EMAIL PROTECTED] writes: On Sun, Nov 30, 2003 at 07:31:09PM -0500, Perry E.Metzger wrote: Although it is still not as stable as we'd like, the benchmarks of the native threads on NetBSD are pretty damn impressive. I'd say that not using the native threads would be a tremendous waste... When NPTL is ported, I assume you mean the Netscape stuff? It is running on top of our libc's pthreads right now in NetBSD. The problem is you don't HAVE our pthreads if you go to glibc. it'll be no difference. The native word is meaningless here. I see you haven't studied the scheduler activations framework. The scheduler activations infrastructure is VERY complicated. It took a couple of years to write and has taken us nearly a year to get the bugs shaken out of it, using all native tools. You would be much better off just specifying what was missing from the native libc so that it could be added -- that, at least, is a tractable problem. -- Perry E. Metzger[EMAIL PROTECTED]
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 01:22:52PM +0100, Robert Millan wrote: Please, how is threading implemented in the NetBSD (and FreeBSD) port of Glibc? The goal is porting NPTL for both KFreeBSD and KNetBSD (K for 'kernel of'). As of now, temporary solutions are being used: linuxthreads and libpth, respectively. Are you sure that's the right way to go? Porting NPTL looks likely to require drastic design changes. NetBSD and FreeBSD kernel support for threads is completely different than on Linux. The design rationale for NPTL is completely different, too. Linux kernel developers rejected M:N threading, while NetBSD and FreeBSD seem to have seen some benefit to it. NPTL is making use of things like kernel support for thread local storage, futexes, and altered or new system calls. None of those exist on BSD kernels, AFAIK. Equivilent facilities will most likely not have compatible semantics, either. NPTL is so tied to Linux 2.6 that I'm not sure you can port it without rewriting its core, or extensive patching of the BSD kernels. I suspect it'd be easier to port libkse to glibc. (And the equivilent on NetBSD.) Libc dependacies are easier to change than kernel dependacies. ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 01:56:04PM -0500, Perry E.Metzger wrote: Robert Millan [EMAIL PROTECTED] writes: On Sun, Nov 30, 2003 at 07:31:09PM -0500, Perry E.Metzger wrote: Although it is still not as stable as we'd like, the benchmarks of the native threads on NetBSD are pretty damn impressive. I'd say that not using the native threads would be a tremendous waste... When NPTL is ported, I assume you mean the Netscape stuff? It is running on top of our libc's pthreads right now in NetBSD. The problem is you don't HAVE our pthreads if you go to glibc. No, he's talking about the new threads library on Linux 2.6. There's a design overview here: http://people.redhat.com/drepper/nptl-design.pdf it'll be no difference. The native word is meaningless here. That last sentence doesn't make sense, Robert. If you do get NPTL running on NetBSD, you'll have to use the same NetBSD kernel interfaces as the native libc does. I see you haven't studied the scheduler activations framework. I think he's ignorant of the scale of the problem, yes. The scheduler activations infrastructure is VERY complicated. It took a couple of years to write and has taken us nearly a year to get the bugs shaken out of it, using all native tools. You would be much better off just specifying what was missing from the native libc so that it could be added -- that, at least, is a tractable problem. I think in some viewpoints (certainly not mine), the problem may be that it's not glibc. But that's just my impression. In any case, it's obviously far saner to try to port libpthreads from the NetBSD libc to glibc than to port NPTL from Linux. ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 01:59:49PM -0500, Perry E. Metzger wrote: Robert Millan [EMAIL PROTECTED] writes: On Sun, Nov 30, 2003 at 09:46:19PM -0500, Perry E.Metzger wrote: IMHO, the amount of work involved in making glibc stably work with scheduler activations is likely prohibitive. You'll be chasing problems in the library forever. First we'll merge the patchset in upstream. Then we'll have problems for a while, similarly to those the GNU/Hurd port has fixing Glibc every time it breaks for them. But unlike GNU/Hurd, at some point upstream developers will install GNU/K*BSD themselves and maintain it for us. If you mean that the NetBSD folks are going to abandon their libc, which is really nice to work with, I think you're mistaken. It is unlikely that they're ever going to do that. (They includes me, fyi.) Because of that, you'll have to maintain patches to do the scheduler activations dance forever. SA is probably the most complicated way to do threads that's out there, so this will not, in the end, be particularly pleasant. If you had a list of functional deficiencies in the native libc, though, it would probably be possible to re-implement them and fix them in the native NetBSD libc. NetBSD would like to be maximally compatible with third party apps, so we add stuff we need all the time and are happy to do it. It would also likely be far less work to add a few dozen new functions to libc than for you to re-implement the userland SA framework and debug it. I can say, from experience, that this is entirely true. Having found a couple of fairly major deficienies (__cxx_atexit, [n]ftw), they're quite willing to work with folks to figure out ways to add support for just about anything that's sane, and discuss what is or is not sane, and why. :) -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgp1U8BZDns02.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
Nathan Hawkins [EMAIL PROTECTED] writes: You would be much better off just specifying what was missing from the native libc so that it could be added -- that, at least, is a tractable problem. I think in some viewpoints (certainly not mine), the problem may be that it's not glibc. But that's just my impression. If one wants Linux, there is Linux, and one doesn't need to do any work. If one wants to marry the advantages of NetBSD with the Debian tools, then getting rid of the interesting things about NetBSD won't achieve the desired result. glibc has never struck me as especially stunningly nifty, but I suppose others have their own tastes. In any case, it's obviously far saner to try to port libpthreads from the NetBSD libc to glibc than to port NPTL from Linux. I'd say it would be impossible to make the new Linux threads work on NetBSD in a reasonable amount of time. That said, the NetBSD threads stuff has pervasive hooks throughout libc -- Perry E. Metzger[EMAIL PROTECTED]
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 01:35:10PM -0500, Nathan Hawkins wrote: First we'll merge the patchset in upstream. Then we'll have problems for a while, similarly to those the GNU/Hurd port has fixing Glibc every time it breaks for them. I'd be more impressed with that analysis if the bugs were in fact Time to wait, then. But unlike GNU/Hurd, at some point upstream developers will install GNU/K*BSD themselves and maintain it for us. How do you know that? Why should they do this when they didn't for the Hurd? Because GNU/K*BSD is stable and full-featured. I'll be able to migrate myself in a matter of weeks. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 01:59:49PM -0500, Perry E.Metzger wrote: If you mean that the NetBSD folks are going to abandon their libc, which is really nice to work with, I think you're mistaken. It is unlikely that they're ever going to do that. (They includes me, fyi.) I'm not impressed. If you had a list of functional deficiencies in the native libc, though, it would probably be possible to re-implement them and fix them in the native NetBSD libc. NetBSD would like to be maximally compatible with third party apps, so we add stuff we need all the time and are happy to do it. It would also likely be far less work to add a few dozen new functions to libc than for you to re-implement the userland SA framework and debug it. Porting to a Glibc-based from another is kids play. In a few months I started two ports and brought them to a stage that took years for a group of people to attain. And I have seen your less work patches already. Xfree86 and pam are good examples. Have a look at them. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 03:10:28PM -0500, Perry E.Metzger wrote: If one wants Linux, there is Linux, and one doesn't need to do any work. If one wants to marry the advantages of NetBSD with the Debian tools, then getting rid of the interesting things about NetBSD won't achieve the desired result. glibc has never struck me as especially stunningly nifty, but I suppose others have their own tastes. This is not about the Debian tools, you can have Debian tools even on NetBSD and get them into the ports collection. It's about the GNU libc and userland, which are the standard in Debian and I see no reason to replace them. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 02:32:03PM -0500, Nathan Hawkins wrote: The goal is porting NPTL for both KFreeBSD and KNetBSD (K for 'kernel of'). As of now, temporary solutions are being used: linuxthreads and libpth, respectively. Are you sure that's the right way to go? Porting NPTL looks likely to require drastic design changes. NetBSD and FreeBSD kernel support for threads is completely different than on Linux. I'm not. But I heard that NPTL is portable and that the Glibc/Hurd/Mach developers intend to use it for Glibc/Hurd/Mach. We can discuss that later though, we're not going to port NPTL yet. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 10:30:20PM +0100, Robert Millan wrote: On Mon, Dec 01, 2003 at 01:35:10PM -0500, Nathan Hawkins wrote: I'd be more impressed with that analysis if the bugs were in fact Time to wait, then. Yes, that's what I've been doing. I've tried to work on glibc, and pretty much concluded it's not something _I_ want to do. So I've been waiting on others to fix it. After about 18 months now, there's little change. But unlike GNU/Hurd, at some point upstream developers will install GNU/K*BSD themselves and maintain it for us. How do you know that? Why should they do this when they didn't for the Hurd? Because GNU/K*BSD is stable and full-featured. I'll be able to migrate myself in a matter of weeks. So someone fixed the bug in the DNS resolver? Without it full-featured is kind of hard to believe, since all kinds of packages are unusable. (Think about email, for example...) If it were really stable and usable, I'd be installing servers with it now. I'd love to. But it isn't. ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 06:00:06PM -0500, Perry E.Metzger wrote: Robert Millan [EMAIL PROTECTED] writes: I'm not impressed. Hey, you said that you would be able to avoid having to deal with maintaining the glibc stuff yourself because the upstream would do it for you. I doubt that the FSF people will do it, and I doubt that we (NetBSD) are going to do it, so what's left? It isn't the FSF who maintains Glibc. As I said, though, it is likely that the NetBSD folks would happily add needed stuff from glibc to the netbsd libc. I mean, who wants to have a libc that won't run popular third party apps? Of course we'd be amenable. Please avoid the third party euphemism. If you want to run non-free software on a Glibc-based system, you can use the NetBSD libc since it's no technical problem to provide it as alternative (ala Linux libc5) Porting to a Glibc-based from another is kids play. You aren't listening. The threads stuff is not kids play. [...] The threads stuff is not as relevant nor as complicated as you pretend. I've been working with libpth (and so has the NetBSD 1.6 libc based port) and haven't found any incompatibility yet. At some point we'll study which of porting NPTL or porting the pthreads from NetBSD's libc is a more viable option. If you want to help or propose a solution, that's fine, but stop playing devil's advocate which is just a waste of my time. In a few months I started two ports and brought them to a stage that took years for a group of people to attain. Ever hear of the 80/20 rule? I easily believe you can get the thing very far along inside a couple of months. Trying to make it actually bulletproof for production use -- especially with all the bells and whistles complete -- is not the same thing. And that doesn't apply to the other port? Well, I think we should postpone the discussion untill the port is ready for production use, in a pair of weeks. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 05:29:37PM -0500, Nathan Hawkins wrote: Because GNU/K*BSD is stable and full-featured. I'll be able to migrate myself in a matter of weeks. So someone fixed the bug in the DNS resolver? Without it full-featured is kind of hard to believe, since all kinds of packages are unusable. (Think about email, for example...) Yep.. I'm working on that. If it were really stable and usable, I'd be installing servers with it now. I'd love to. But it isn't. That'll be soon I hope. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
Robert Millan [EMAIL PROTECTED] writes: I'm not impressed. Hey, you said that you would be able to avoid having to deal with maintaining the glibc stuff yourself because the upstream would do it for you. I doubt that the FSF people will do it, and I doubt that we (NetBSD) are going to do it, so what's left? It isn't the FSF who maintains Glibc. Whatever. The point is the maintainers won't do the work for you, so the upstream comment doesn't hold much water. As I said, though, it is likely that the NetBSD folks would happily add needed stuff from glibc to the netbsd libc. I mean, who wants to have a libc that won't run popular third party apps? Of course we'd be amenable. Please avoid the third party euphemism. If you want to run non-free software on a Glibc-based system, That's not what third party means. Third party means stuff not provided by The NetBSD Foundation in our releases. The BSD world doesn't work quite the way the Linux world does in this regard. We maintain both a kernel and a tightly integrated userland that goes with it -- anything else, for example, Gnu Emacs, is third party. Our pkgsrc infrastructure exists to make it easy to compile third party software, but we do not claim that Emacs and /bin/ls are supported the same way. We've got about 4500 packages in pkgsrc -- a fraction of the number some folks like Debian support, but quite a number -- and in the course of making them all work we routinely find that we have to fix things in NetBSD. For example, programs like xmms have inspired many changes to our threads system. Anyway, my statement was very straightforward -- if there are interfaces that apps need that they don't have, we would obviously want them in NetBSD's libc so we can run those apps. I don't doubt that there are things our libc lacks (and that glibc lacks for that matter) and it would be reasonable to fix them. We're interested in being as functional and compatible as possible. Porting to a Glibc-based from another is kids play. You aren't listening. The threads stuff is not kids play. [...] The threads stuff is not as relevant nor as complicated as you pretend. I've been working with libpth pth is not a real preemptive threads library. It doesn't work very well as a result. You need kernel support to do threads right, and the kernel support is necessarily intimately connected to the userland implementation that interfaces with it. At some point we'll study which of porting NPTL or porting the pthreads from NetBSD's libc is a more viable option. You will never port NPTL, period, because to do that would require that you change the NetBSD kernel, and you aren't in a position to do that. Porting the pthreads from NetBSD's libc will be a very serious job, and as I noted will require tracking all the changes that get made in NetBSD's pthreads engine on a constant basis, because as I said, there is a protocol that the kernel and userland parts of the pthreads implementation have to agree on. If you make any mistakes in altering routines in glibc, since you're dealing with threads stuff it may take months or years to find all the heisenbugs that locking code brings in. Ditto, by the way, for bugs in the thread engine itself. If you want to help or propose a solution, that's fine, but stop playing devil's advocate which is just a waste of my time. I'm not playing devil's advocate. I'm trying to tell you the truth to try to help you. You seem to be unwilling to listen. Thus, I'll drop out of the conversation. Do whatever you want. -- Perry E. Metzger[EMAIL PROTECTED]
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 10:56:30PM +0100, Robert Millan wrote: On Mon, Dec 01, 2003 at 03:10:28PM -0500, Perry E.Metzger wrote: If one wants Linux, there is Linux, and one doesn't need to do any work. If one wants to marry the advantages of NetBSD with the Debian tools, then getting rid of the interesting things about NetBSD won't achieve the desired result. glibc has never struck me as especially stunningly nifty, but I suppose others have their own tastes. This is not about the Debian tools, you can have Debian tools even on NetBSD and get them into the ports collection. It's about the GNU libc and userland, which are the standard in Debian and I see no reason to replace them. For the record, probably 70% of the email that I've seen on this list falls somewhere under the category of a single, perpetual, floating discussion/argument/flamewar. To wit: NetBSD/FreeBSD/OpenBSD? GNU libc or native libc? (I've never seen another alternative, IIRC) GNU userland or native userland? These have never been solved; there are salient points of argument for and against each. In the time-tested tradition, we've had a bunch of time spent on scattered development efforts, some of which have made it further than others. Many of them have caused improvements to Debian as a whole, and not infrequently to upstream software. I believe my preferences are well established, and if anyone is in doubt of the rationale, I'm happy to explain it - in private email. I would say that the general GNU userland is standard to Debian, in that people expect the 'ls' to be GNU ls, and similar things. I am vastly more hesitant to ascribe the GNU libc the same status, entirely because users *do not* normally see it, in any fashion, and the very GNU userland that is Debian's default is entirely capable of working with other libc packages just fine. But then, I suppose that may come of my actual goal, which pretty much boils down to: I want to feel like I'm running a Debian system, and be able to administer it with the same commands as any other Debian system, but I want to have a choice about what's under the hood, in case I have technical reasons to prefer something other than Linux + Glibc. (In fact, I even have some fairly real-world situations in mind that would vastly benefit from have neither of the two, which is why I cared enough to start working on it, origionally; they're less relevant, now, since I no longer work at that job). -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpRhKY5Mn7TY.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 01:30:55AM +0100, Robert Millan wrote: On Mon, Dec 01, 2003 at 05:18:29PM -0500, Nathan Hawkins wrote: Also, I'd like to point out to you that when this started, none of the people involved had any real experience with porting Debian. So that learning curve, plus RL demands on our time, had a lot to do with the time it took to accomplish anything. Anyway, I seem to recall that the critical part of my work on the native libc port got done in about 2-3 months. I recall having participated in both ports years ago. I do also recall seeing GNU/*BSD systems with *BSD libc based on _slink_. Seems obvious that progress is very slow with *BSD libc's. And, last I knew, pretty much all of that work got thrown out the window because it had suffered serious bitrot as Debian (and the BSDs) moved forward, while nobody was hammering on it actively. I know that the vast majority of my work was done in intermittent bursts of about a month each, as life allowed them. And I have seen your less work patches already. Xfree86 and pam are good examples. Have a look at them. We fixed pam to run on native libc a long time ago. It wasn't that bad, once I got libshadow written. And last I knew you didn't have an X server package, which I had on the native libc a long time ago. I was referring to the GNU/NetBSD port. See bug #201683 for example, and compare it to the one-liner patch I sent to pam. As for Xfree86, try wc -l debian/patches/84* in the source tree. Just a pair of examples. (btw, fixing the X server is on my todo) All I have to say about the X server, as the person who generated most of the patches, is that they're actually very straightforward, if rather invasive. I simply had to go through each config option and decide whether it should be handled in the 'native' way, or the GNU-userland way (and it was very much a userland issue, not a libc issue). The *hard* part was in hunting down build problems and bad assumptions in something the size of the X codebase. That isn't going to be any saner on a Glibc+FreeBSD system; probably less sane, in fact. -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpb4PR9Jc3Re.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 01:52:01AM +0100, Robert Millan wrote: On Mon, Dec 01, 2003 at 06:00:06PM -0500, Perry E.Metzger wrote: As I said, though, it is likely that the NetBSD folks would happily add needed stuff from glibc to the netbsd libc. I mean, who wants to have a libc that won't run popular third party apps? Of course we'd be amenable. Please avoid the third party euphemism. If you want to run non-free software on a Glibc-based system, you can use the NetBSD libc since it's no technical problem to provide it as alternative (ala Linux libc5) He's referring to source compatibility. On BSD's, third party seems to mostly mean just about anything not in the base system. NetBSD has binary emulation for Linux. That takes care of the non-free stuff, since that's far more available on Linux than for NetBSD. What he's offering is to look at patches that bring NetBSD's libc into source compatibility with glibc. You aren't listening. The threads stuff is not kids play. [...] The threads stuff is not as relevant nor as complicated as you pretend. I've been working with libpth (and so has the NetBSD 1.6 libc based port) and haven't found any incompatibility yet. At some point we'll study which of porting NPTL or porting the pthreads from NetBSD's libc is a more viable option. No, _you_ are underestimating the complexities. There is a very large difference between something like libpth or linuxthreads (which are either entirely based in userspace, or require minimal support from the kernel), and the very kernel-specific implementations being used in NPTL, FreeBSD's libkse or NetBSD's libpthreads. The newer threads packages are targeting drastic improvements in performance, Posix compliance, stability, and scalability. And from what I understand, they're getting them. Unfortunately, those gains come at the expense of dramatically more complex and incompatible kernel support. And the BSD mechanisms aren't much like the ones Linux uses. There's some useful documentation on how threads work on FreeBSD here: http://www.freebsd.org/kse/ And there's a pretty good paper about NetBSD threads here: http://web.mit.edu/nathanw/www/usenix/freenix-sa/freenix-sa.html I'm sure others can provide more or better references. ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
On Tue, Dec 02, 2003 at 01:52:01AM +0100, Robert Millan wrote: On Mon, Dec 01, 2003 at 06:00:06PM -0500, Perry E.Metzger wrote: Porting to a Glibc-based from another is kids play. You aren't listening. The threads stuff is not kids play. [...] The threads stuff is not as relevant nor as complicated as you pretend. I've been working with libpth (and so has the NetBSD 1.6 libc based port) and haven't found any incompatibility yet. At some point we'll study which of porting NPTL or porting the pthreads from NetBSD's libc is a more viable option. If you want to help or propose a solution, that's fine, but stop playing devil's advocate which is just a waste of my time. *bzzzt* The NetBSD/native port has been stalled for some time, because I ran into core, required-to-build-lots-of-things applications (tcl8.4, IIRC, in particular) that *don't work* with GNU pth. Period. Neither version 1.x or 2.x. Speaking as someone who has co-written SA stuff (granted, for toy operating systems that were proof-of-concept, not enterprise support), it is neither trivial nor easy, and speaking as someone who has patched the innards of both NetBSD's kernel and libc, and who has to figure out policy for Debian packages to not make it easy for users to end up with unbootable systems, the kernel and the libc are deeply intertwined, and in few places moreso than the threading stuff. Remember, Glibc *is* the Linux libc, these days; that means that folks put a lot of effort into ensuring that it will support Linux, because otherwise no Linux system can run. The motivation to port such a complex part of it to an OS whose *fundamental assumptions* about threading are *fundamentally different* - 1:1 vs M:N is about as different as it gets - seems a bit iffy, to me. While I'd dearly love to see a bit more de-coupling of NetBSD kernel and libc (so that they don't have to be in quite such lockstep, though I'm more worried about the process utilities that must be *exact* matches), I don't claim that managing it would be trivial, and may not even be practical (or at least, might not be feasible short of 3.0). In a few months I started two ports and brought them to a stage that took years for a group of people to attain. Ever hear of the 80/20 rule? I easily believe you can get the thing very far along inside a couple of months. Trying to make it actually bulletproof for production use -- especially with all the bells and whistles complete -- is not the same thing. And that doesn't apply to the other port? Well, I think we should postpone the discussion untill the port is ready for production use, in a pair of weeks. I'll be happy to write some applications which (legitimately) use hundreds of threads at a time, and load-test it for you. Or, if you'd rather, I can do some scientific software that can fire off as many thousand threads as you'd like, all vying for the same shared mmaped memory. :) I know NPTL on Linux handles this, and I'm reasonably confident that native threads on NetBSD -current will handle it as well. Until your port can handle it, I wouldn't trust it enough to call it threadsafe (and yes, that means no version of Linux prior to NPTL is threadsafe, in my opinion - having had to deal with it in such a context, I'm quite certain it *isn't*). -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpa8AEaKrBxh.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
We fixed pam to run on native libc a long time ago. It wasn't that bad, once I got libshadow written. And last I knew you didn't have an X server package, which I had on the native libc a long time ago. I was referring to the GNU/NetBSD port. See bug #201683 for example, and compare it to the one-liner patch I sent to pam. As for Xfree86, try wc -l debian/patches/84* in the source tree. Just a pair of examples. (btw, fixing the X server is on my todo) All I have to say about the X server, as the person who generated most of the patches, is that they're actually very straightforward, if rather invasive. I simply had to go through each config option and decide whether it should be handled in the 'native' way, or the GNU-userland way (and it was very much a userland issue, not a libc issue). The *hard* part was in hunting down build problems and bad assumptions in something the size of the X codebase. That isn't going to be any saner on a Glibc+FreeBSD system; probably less sane, in fact. That was pretty much my experience, too. In fact, I tried to get it working on glibc, and had fits with it. I particularly remember xterm being a disaster. I gave up on it, and Robert evidently got it working except for the server. It probably needs some headers in sys/ that glibc didn't get right. ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 09:26:33PM -0500, Perry E. Metzger wrote: We've got about 4500 packages in pkgsrc -- a fraction of the number some folks like Debian support, but quite a number -- and in the course of making them all work we routinely find that we have to fix things in NetBSD. For example, programs like xmms have inspired many changes to our threads system. Like I said, [n]ftw is one example. Debian's dpkg sources require it, and (once given a suitable proposed code) the NetBSD folks have been very good about hammering on it to get the bugs out, and include it in their libc. It's also an excellent example of why I, personally, want to avoid ever using Glibc again if I have a choice - look at the code I wrote for NetBSD's use, and then look at the code in Glibc to do the same stuff. Then look for some borax and a skull drill, to get the horror out of your brain. -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpaXMyeIu3PJ.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
On Mon, Dec 01, 2003 at 08:24:41PM -0700, Joel Baker wrote: On Mon, Dec 01, 2003 at 09:26:33PM -0500, Perry E. Metzger wrote: We've got about 4500 packages in pkgsrc -- a fraction of the number some folks like Debian support, but quite a number -- and in the course of making them all work we routinely find that we have to fix things in NetBSD. For example, programs like xmms have inspired many changes to our threads system. Like I said, [n]ftw is one example. Debian's dpkg sources require it, and (once given a suitable proposed code) the NetBSD folks have been very good about hammering on it to get the bugs out, and include it in their libc. It's also an excellent example of why I, personally, want to avoid ever using Glibc again if I have a choice - look at the code I wrote for NetBSD's use, and then look at the code in Glibc to do the same stuff. Yes, I've had the same experience with utmp and shadow support. (The SUSv2 getutxent and getspent API's.) My code is fairly easy to read, while glibc's is clear as mud. Unfortunately, I haven't had much luck interesting anyone in merging it. Possibly because of compatibility issues, and possibly because the specs for both of those things are kind of icky. Then again, it might simply have had to do with the development cycle for FreeBSD. Then look for some borax and a skull drill, to get the horror out of your brain. That's a pretty good description of how I felt after I started to guess how glibc's weird directory inheritance source structure works. ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
Joel Baker [EMAIL PROTECTED] writes: Please, how is threading implemented in the NetBSD (and FreeBSD) port of Glibc? On the FreeBSD port, it uses linuxthreads. It isn't going to be good enough long term. The NetBSD port of glibc will probably do the same, at least initially. I'm not sure if it's that far along yet, though. *shudder* For the record, one of the major reasons for updating the netbsd kernel sources to -current is to pull in the threading support that will be released with 2.0 (some packages simply *will not* build with GNU pth as pthreads). Although it is still not as stable as we'd like, the benchmarks of the native threads on NetBSD are pretty damn impressive. I'd say that not using the native threads would be a tremendous waste... -- Perry E. Metzger[EMAIL PROTECTED]
Re: Glibc-based Debian GNU/KNetBSD
On Sun, Nov 30, 2003 at 07:31:09PM -0500, Perry E.Metzger wrote: Joel Baker [EMAIL PROTECTED] writes: Please, how is threading implemented in the NetBSD (and FreeBSD) port of Glibc? On the FreeBSD port, it uses linuxthreads. It isn't going to be good enough long term. The NetBSD port of glibc will probably do the same, at least initially. I'm not sure if it's that far along yet, though. *shudder* For the record, one of the major reasons for updating the netbsd kernel sources to -current is to pull in the threading support that will be released with 2.0 (some packages simply *will not* build with GNU pth as pthreads). Although it is still not as stable as we'd like, the benchmarks of the native threads on NetBSD are pretty damn impressive. I'd say that not using the native threads would be a tremendous waste... For my part, I'm of the opinion that threads support based on the native thread libraries is the way to go. I say based on, because it's unlikely that the native threads libraries will work with glibc without at least some porting. However, I do believe it would certainly be better to go that route than to use the linuxthreads port. In FreeBSD's case, the native thread support is very much under development right now. FreeBSD 5.x has three native thread libraries, not counting the port of linuxthreads, which also runs with the native libc. IMHO, the glibc port to FreeBSD is really only about half done. And it was the easy half. There are still issues, and thread support is one of them. Either someone will work on them, or not. If not, I can always restart the native libc port... ---Nathan
Re: Glibc-based Debian GNU/KNetBSD
Nathan Hawkins [EMAIL PROTECTED] writes: Although it is still not as stable as we'd like, the benchmarks of the native threads on NetBSD are pretty damn impressive. I'd say that not using the native threads would be a tremendous waste... For my part, I'm of the opinion that threads support based on the native thread libraries is the way to go. I say based on, because it's unlikely that the native threads libraries will work with glibc without at least some porting. However, I do believe it would certainly be better to go that route than to use the linuxthreads port. IMHO, the amount of work involved in making glibc stably work with scheduler activations is likely prohibitive. You'll be chasing problems in the library forever. Perry
Re: Glibc-based Debian GNU/KNetBSD
On Thu, Nov 20, 2003 at 11:12:35AM +1100, matthew green wrote: A usable Debian GNU/KNetBSD system based on Glibc is now available: so... can someone explain what the plan is for deb/netbsd wrt libc? ie, will there be 2 systems one that is compatible with netbsd (ie that people can upgrade to, run normal netbsd apps, etc) and one that isn't (ie, glibc) ? or will the binary compatible debian/netbsd die now that glibc version has arrived? I have no intent of switching to the GNU libc version; not an impugnment of his work, mind you, but one of the main reasons I bother with NetBSD is so that I can have the native libc (and it's attendant lack of bloat). We get further and further along the path of dammit, we really need to specify architectures by a kernel/arch/libc triple :/ I'm back from Thailand, now, BTW, and will be working to get the new colo machine for debian-bsd.lightbearer.com set up in the fairly immediate future. -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpQHVlru54gs.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
Hello, Hi! A usable Debian GNU/KNetBSD system based on Glibc is now available: http://people.debian.org/~rmh/knetbsd/pub/ I got to port all the base system with very little effort, since my previous patches for Glibc-based GNU/KFreeBSD already did most of the job. Xfree86 and Please, how is threading implemented in the NetBSD (and FreeBSD) port of Glibc? Bye Pavel
Re: Glibc-based Debian GNU/KNetBSD
On Sat, Nov 29, 2003 at 01:40:12PM -0500, Nathan Hawkins wrote: On Sat, Nov 29, 2003 at 03:45:15PM +0100, Pavel Cahyna wrote: Please, how is threading implemented in the NetBSD (and FreeBSD) port of Glibc? On the FreeBSD port, it uses linuxthreads. It isn't going to be good enough long term. The NetBSD port of glibc will probably do the same, at least initially. I'm not sure if it's that far along yet, though. *shudder* For the record, one of the major reasons for updating the netbsd kernel sources to -current is to pull in the threading support that will be released with 2.0 (some packages simply *will not* build with GNU pth as pthreads). -- Joel Baker [EMAIL PROTECTED],''`. Debian GNU NetBSD/i386 porter: :' : `. `' `- pgpfGJgXy1luP.pgp Description: PGP signature
Re: Glibc-based Debian GNU/KNetBSD
[ Please keep the CC ] On Thu, Nov 20, 2003 at 11:12:35AM +1100, matthew green wrote: A usable Debian GNU/KNetBSD system based on Glibc is now available: so... can someone explain what the plan is for deb/netbsd wrt libc? ie, will there be 2 systems one that is compatible with netbsd (ie that people can upgrade to, run normal netbsd apps, etc) and one that isn't (ie, glibc) ? or will the binary compatible debian/netbsd die now that glibc version has arrived? Frankly, I don't know. But I can tell you I'll keep working on the Glibc-based port since it produced good results so far. I've done something with Glibc in months, that took years to produce with NetBSD libc. We'll get to discuss that at some point. For now, I invite everyone to try the Glibc-based system and see the results. -- Robert Millan [..] but the delight and pride of Aule is in the deed of making, and in the thing made, and neither in possession nor in his own mastery; wherefore he gives and hoards not, and is free from care, passing ever on to some new work. -- J.R.R.T, Ainulindale (Silmarillion)
Re: Glibc-based Debian GNU/KNetBSD
On Thu, Nov 20, 2003 at 11:12:35AM +1100, matthew green wrote: A usable Debian GNU/KNetBSD system based on Glibc is now available: so... can someone explain what the plan is for deb/netbsd wrt libc? ie, will there be 2 systems one that is compatible with netbsd (ie that people can upgrade to, run normal netbsd apps, etc) and one that isn't (ie, glibc) ? or will the binary compatible debian/netbsd die now that glibc version has arrived? I would like to point out to you that you can retain binary compatibility. I did it with FreeBSD by moving the libraries into a different directory (/libfbsd and /usr/libfbsd). A very trivial hack to the dynamic linker, and native binaries work fine, even in /usr/bin. Alternatively, you can always have a chroot. That's not at all difficult. With FreeBSD, it's possible to install one using a shell script that's right on the installation media. ---Nathan
re: Glibc-based Debian GNU/KNetBSD
A usable Debian GNU/KNetBSD system based on Glibc is now available: so... can someone explain what the plan is for deb/netbsd wrt libc? ie, will there be 2 systems one that is compatible with netbsd (ie that people can upgrade to, run normal netbsd apps, etc) and one that isn't (ie, glibc) ? or will the binary compatible debian/netbsd die now that glibc version has arrived? thanks, .mrg.
