Re: FreeBSD 7, DragonFly's status
Rahul Siddharthan [EMAIL PROTECTED] writes: I'm still keeping an eye on DragonFly, and hope to run it again one of these days. I think the biggest problem in FreeBSD that DragonFly fixes is attitude. It's not a completely accurate answer, but attitude is certainly sufficient in my book. I find your comment very appropriate. In fact, a careful observer might find this as a primary reason people are still using this system despite these apparent shortcomings of SMP, 64-bit, etc. In my experience, technical issues are much easier to fix than collective attitudes. I think all DragonFly really needs is more good developers, and this appears to be happening at the correct speed to preserve the fresh attitude this community currently has. As to the original posting, technical comparison is normally a good thing. However: Dmitri Nikulin [EMAIL PROTECTED] writes: I'm gravely concerned about the growth and survival of this brilliant project in the face of increasing pressure from projects with much larger developer communities and software ecosystems. Exactly what pressure are you referring to here? :) -- Dave Hayes - Consultant - Altadena CA, USA - [EMAIL PROTECTED] The opinions expressed above are entirely my own Every stick has two ends.
Re: FreeBSD 7, DragonFly's status
On Thu, Feb 28, 2008 at 4:09 PM, Justin C. Sherrill [EMAIL PROTECTED] wrote: On Wed, February 27, 2008 11:29 pm, Dmitri Nikulin wrote: The benchmark at http://people.freebsd.org/~kris/scaling/os-mysql.png (for the full presentation, see http://www.freedomtc.com/pdf/7.0_Preview.pdf, that plot is on slide 17) indicates that FreeBSD 7 not only competes strongly with current Linux performance and scalability, but that DragonFly has been beaten even by NetBSD which came late to the SMP party. Minor quibble: you're pointing at benchmarks on an 8-core xeon. Relatively uncommon hardware, though I'm sure that'll change within a year or so. Sure, but SMP scalability is one of the key goals of DragonFly, and for it to be beaten by NetBSD (for which this is not a major goal, and for which SMP scalability only started being worked on a year ago) is very confusing. I haven't seen it compared with 1.12, but since no huge scalability work has gone in for a long time, I doubt it would close the gap much. Even so, while 8-core Xeons are not yet common, AMD64 compatible chips are pretty much the only mass consumer chips available now, and DragonFly can still only use the 32 bit portion of that, and even then, doesn't yet scale to two cores fully, let alone the increasingly common quad cores. That falls far short of the goal to scale well for increasingly parallel systems, which continue to be picked up by Linux and now FreeBSD at an increasing rate. That's really disappointing for an operating system I originally believed would humiliate FreeBSD's SMP implementation. At the risk of sounding like a troll, may I ask, if FreeBSD 7 has high performance, high stability and remains reasonably clean and maintainable, doesn't that partly invalidate the reasons DragonFly was created? Being cleaner and more revolutionary doesn't count for much if the product itself doesn't serve as a compelling alternative. Maybe I'm just missing the point of DragonFly's development. There were people attracted to DragonFly early on because they were mistreated by various folks involved with FreeBSD, but liking DragonFly because it's not FreeBSD is not enough to build a project. DragonFly is not a FreeBSD replacement, nor does DragonFly require FreeBSD to suck in order to exist. You could probably make the exact same argument you just did but substitute in the name of another BSD, if you are going to take the tack of fastest benchmark so why use anything else?. (I realize I'm oversimplifying your words.) You're right - you are oversimplifying my words. I'm only saying that, for goals which DragonFly considers groundwork, FreeBSD still matches or wins by a large margin. That includes good performance (FreeBSD wins) and stability (FreeBSD seems to be no worse off). I can't attest to the maintainability, but they seem to be doing alright, and cleanups get done too. So while DragonFly has a lot going for it (which I never denied), it's challenging to find a situation for which DragonFly is actually a better choice than FreeBSD. DragonFly is oriented towards creating a single system image over multiple networked computers on the Internet, which has never been done before. It's years off even for DragonFly. If it does one day achieve that goal, but by then other systems have advanced to the point that DragonFly isn't even compelling for a single node, why will anyone want to run it just to join a cluster? It needs some minimum user base before internet clustering becomes useful at all. That's my biggest fear regarding this project - that by the time its highest goals are achievable, the full potential of those goals will still be out of reach, perhaps forever. FreeBSD - run on x86 OpenBSD - run securely NetBSD - run on anything DragonFly - run once everywhere? For myself, I find DragonFly useful because it's a tight and friendly developer community, with lots of interesting projects to do. FreeBSD having a good mysql benchmark doesn't affect that. neglected. Again, at the risk of sounding like a troll, I'm gravely concerned about the growth and survival of this brilliant project in the face of increasing pressure from projects with much larger developer communities and software ecosystems. We've been doing pretty well, really. The number of changes and new developers in 2007 have been on an upswing relative to 2006. (I should know, given my digest work.) There's been enough happening that I've had a steady 2-day buffer of things to post for a month, which has never happened before. That's really good to hear. It's hard to tell just from mailing lists. See, my problem is, I really care about DragonFly from a very geeky perspective. I want the clever, somewhat revolutionary ideas to win and show the rest of the world how it's done. But on the other hand, Inferno was set to do that, and how relevant is that now? -- Dmitri Nikulin
Re: FreeBSD 7, DragonFly's status
On Thu, 28 Feb 2008 21:53:36 +1100 Dmitri Nikulin [EMAIL PROTECTED] wrote: On Thu, Feb 28, 2008 at 4:09 PM, Justin C. Sherrill [EMAIL PROTECTED] wrote: On Wed, February 27, 2008 11:29 pm, Dmitri Nikulin wrote: The benchmark at http://people.freebsd.org/~kris/scaling/os-mysql.png (for the full presentation, see http://www.freedomtc.com/pdf/7.0_Preview.pdf, that plot is on slide 17) indicates that FreeBSD 7 not only competes strongly with current Linux performance and scalability, but that DragonFly has been beaten even by NetBSD which came late to the SMP party. Minor quibble: you're pointing at benchmarks on an 8-core xeon. Relatively uncommon hardware, though I'm sure that'll change within a year or so. Sure, but SMP scalability is one of the key goals of DragonFly, and for it to be beaten by NetBSD (for which this is not a major goal, and for which SMP scalability only started being worked on a year ago) is very confusing. I haven't seen it compared with 1.12, but since no huge scalability work has gone in for a long time, I doubt it would close the gap much. AIUI not using the approach to SMP that FreeBSD used was a major factor in starting DragonFly on the basis that it hurts single proceesor performance and is inelegant. I'm sure the DragonFly team would welcome a developer determined to push the BGL steadily further inwards until it vanishes. We would then all find out if the DragonFly approach to SMP really is the best. Until that work is done there is no real way of telling. So while DragonFly has a lot going for it (which I never denied), it's challenging to find a situation for which DragonFly is actually a better choice than FreeBSD. I really like the continuous rewindable backups I get from the journal system in DragonFly, AFAIK that's still unique to DragonFly. I look forward to better ones with Hammer. All my boxes are uniprocessor so SMP performance really doesn't matter at all to me, but losing data does :) -- C:WIN | Directable Mirror Arrays The computer obeys and wins.| A better way to focus the sun You lose and Bill collects. |licences available see |http://www.sohara.org/
Re: DragonFly 1.12 Released!
Petr Janda wrote: The DragonFly live cd doesn't have a GUI. If you want to test DragonFly you should install it and then use pkgsrc to install GUI. Petr OK. Thanks! Best regards Sten Solberg
Re: FreeBSD 7, DragonFly's status
On Thu, February 28, 2008 5:53 am, Dmitri Nikulin wrote: before internet clustering becomes useful at all. That's my biggest fear regarding this project - that by the time its highest goals are achievable, the full potential of those goals will still be out of reach, perhaps forever. Only if someone else figures out an open-source SSI system first. I'm not so worried about this because open source operating systems aren't really important any more. People care about the applications running on them. People are far more concerned about a system's ability to match their application environment than anything else these days. Putting it another way: if we couldn't reliably run Apache, we'd be much more worried than if we're relatively low on a benchmark. We've got a big leg up for having pkgsrc and having so many pkgsrc packages run well on DragonFly. (much credit to Joerg for that.) Anyway, this is an open-source project. We don't have to meet any goals or profit levels unless we want to; the only thing we need to do is enjoy what we are doing. Judging by other people's responses here, that's working. That's really good to hear. It's hard to tell just from mailing lists. See, my problem is, I really care about DragonFly from a very geeky perspective. I want the clever, somewhat revolutionary ideas to win and show the rest of the world how it's done. But on the other hand, Inferno was set to do that, and how relevant is that now? I'm curious to see what the commit frequency is over time for DragonFly; I was hoping Ohloh would have that tracked, but I don't see it. There's other folks out there that have solutions graphing from CVS, but nothing yet that's a drop-in solution. ( http://www.ohloh.net/projects/7261/analyses/latest ) Commit frequency isn't an exact measure, but I love infoporn as much as any other geek.
Re: FreeBSD 7, DragonFly's status
Sure, but SMP scalability is one of the key goals of DragonFly, and for it to be beaten by NetBSD (for which this is not a major goal, and for which SMP scalability only started being worked on a year ago) is very confusing. I haven't seen it compared with 1.12, but since no huge scalability work has gone in for a long time, I doubt it would close the gap much. While a scalable SMP implementation is a goal of DragonFly, so is many other things. There isn't very many people around besides Matt that have the time and the skill to help pushing the big giant lock out. it simple as that. Im sure more developers would be welcome. But if evrything goes as planned with HAMMER, i think you'll be seeing more interest in DragonFly quite soon and probably much higher SMP performance by the end of the year. Petr
laptop lcd vs external lcd panel Xorg
Hi, I installed 1.12.0 on my laptop along with 1.10.1 packages (should the packages really be recompiled or can I use 1.10.1 binary packages?). I use an external lcd panel since laptop screen is too small. Now when booting, I switch the screen output to go to the external lcd panel only (ie laptop screen off, all screen/graphics output goes to external panel) so when Xorg starts I can use high resolutions. In 1.12.0 however, when I do `startx', Xorg resets both panels so graphics output is to both panels. This means I can use only low resolution in graphics mode since Xorg chooses the mode suitable for the small laptop lcd. I wonder what has changed in the transition 1.10.1 -- 1.12 so that Xorg now resets screen? Can I disallow Xorg turning on both panels? A sysctl setting or Xorg conf setting? thanks, Must --
Re: FreeBSD 7, DragonFly's status
Well, I'll give you my 5-second opinion. What I am not worried about: * Developer interest has always increased slowly and continues to do so. I'd be interested in commit statistics but my gut feeling, from NOT having to push into subsystems that I used to have to push into, is that more developers are working on more of the critical infrastructure. * More developers are doing the 'harder' aspects of kernel development rather then just me. This is very, very important. * Ports and packages. This was a huge worry of mine at the beginning of the project. I no longer worry about it. What I am worried about: * The big-ticket items that we have traditionally compared ourselves against, SMP primarily, have developed slowly, primarily because up until recently I was the only person working on it. It an issue of man-hours more then anything else. FreeBSD simply has more developers working on SMP then we do. We have the infrastructure and the abstraction, we now need to get down to the brass balls and finish it. * I really want to see someone take up the ball on getting the network stack MP safe. It's important to the project but I can't do that and HAMMER at the same time. It's the easiest thing to make MP safe in the project. * Similarly with AMD64. We need it. I've developed the infrastructure separation required and we even have a fully virtualized kernel (vkernel) which demonstratres the infrastructure separation. Most of the generic kernel code can easily support a 64 bit architecture. The compiler supports it. It's a project waiting for someone to take up the ball. * Our interrupt routing subsystem really needs a major upgrade. (i.e. a major port from FreeBSD). Where I think the future is: * SMP, Storage, and SSI. Real time mirroring at the logical level. HAMMER is a major component for the storage, SSI, and mirroring components, and I believe HAMMER will be a large interest magnet. Our project goals have not changed, but if I had it all to do over again I would have started work on HAMMER much earlier then I did. I spent more time then I should have perfecting the low level infrastructure, trying to build a base upon which all the other work could occur. -Matt
Re: laptop lcd vs external lcd panel Xorg
On Thu, February 28, 2008 12:23 pm, mustkaru wrote: Hi, I installed 1.12.0 on my laptop along with 1.10.1 packages (should the packages really be recompiled or can I use 1.10.1 binary packages?). I I don't have a good answer on the xorg difference between releases, but: most/all of the 1.10.1 packages should work, as I understand it. We'll have 1.12 binary packages soon; I have to get through a build on pkgbox.
Re: laptop lcd vs external lcd panel Xorg
On Thu, Feb 28, 2008 at 6:38 PM, Justin C. Sherrill [EMAIL PROTECTED] wrote: On Thu, February 28, 2008 12:23 pm, mustkaru wrote: Hi, I installed 1.12.0 on my laptop along with 1.10.1 packages (should the packages really be recompiled or can I use 1.10.1 binary packages?). I I don't have a good answer on the xorg difference between releases, but: most/all of the 1.10.1 packages should work, as I understand it. We'll have 1.12 binary packages soon; I have to get through a build on pkgbox. Thanks, am looking forward to 1.12 bin packages! Could you possibly send an announcement to the list too? cheers, M --
Re: laptop lcd vs external lcd panel Xorg
On Thu, February 28, 2008 1:50 pm, mustkaru wrote: Thanks, am looking forward to 1.12 bin packages! Could you possibly send an announcement to the list too? I will - it took I think 8 days to get through the first build for 1.10, so it'll be a little while.
Re: FreeBSD 7, DragonFly's status
Matthew Dillon wrote: * Similarly with AMD64. We need it. I've developed the infrastructure separation required and we even have a fully virtualized kernel (vkernel) which demonstratres the infrastructure separation. Most of the generic kernel code can easily support a 64 bit architecture. The compiler supports it. It's a project waiting for someone to take up the ball. Note that Noah Yan has been working on an AMD64 port. Some stuff is already committed, and last time I checked, he had about 328KB of uncommitted patches in his git repository (http://repo.or.cz/w/dragonfly/port-amd64.git). He doesn't seem to have much time at the moment but said he'll return to it as soon as the stuff he's occupied with has settled down a bit. Sascha -- http://yoyodyne.ath.cx
pkgsrc distfiles (was Re: laptop lcd vs external lcd panel Xorg)
I don't have a good answer on the xorg difference between releases, but: most/all of the 1.10.1 packages should work, as I understand it. We'll have 1.12 binary packages soon; I have to get through a build on pkgbox. That reminds me. Maybe the pbulk or whatever you are using allows you to set your distfiles to be NFS mounted /archive/distfiles. DISTDIR?= /archive/distfiles (Feel free to make a pkgsrc distfiles group or something for that. I don't care if you change the directory or file ownerships.) My only concern was having duplicate downloads of thousands of distfiles. Also to use other locations (to read but not write): DIST_PATH=/build/pbulk_chroot/distfiles:/build/pbulk_chroot_head/distfiles You can read about that with: bmake help topic=distdir Jeremy C. Reed
Re: FreeBSD 7, DragonFly's status
Matthew Dillon wrote: Well, I'll give you my 5-second opinion. *snip* * Our interrupt routing subsystem really needs a major upgrade. (i.e. a major port from FreeBSD). Given that theirs has choked several times on some fairly common hardware that DID work thru 6.2 RELEASE, that would not necessarily be the first or best place to look. Where I think the future is: * SMP, Storage, and SSI. Real time mirroring at the logical level. HAMMER is a major component for the storage, SSI, and mirroring components, and I believe HAMMER will be a large interest magnet. Our project goals have not changed, but if I had it all to do over again I would have started work on HAMMER much earlier then I did. I spent more time then I should have perfecting the low level infrastructure, trying to build a base upon which all the other work could occur. -Matt It may seem so in the rear-view mirror, but had you NOT done the low-level infrastructure, AND the 2+ year code-clean-up of what was adapted from Free (and other) BSD, nothing else would be working as well as it does. That was time that pays back with long-running and ongoing dividends. JM2CW, Bill
Re: FreeBSD 7, DragonFly's status
On Thu, Feb 28, 2008 at 07:13:25PM +, Bill Hacker wrote: Matthew Dillon wrote: I spent more time then I should have perfecting the low level infrastructure, trying to build a base upon which all the other work could occur. -Matt It may seem so in the rear-view mirror, but had you NOT done the low-level infrastructure, AND the 2+ year code-clean-up of what was adapted from Free (and other) BSD, nothing else would be working as well as it does. That was time that pays back with long-running and ongoing dividends. JM2CW, Bill I agree. I really appreciate that you spent the time perfecting the low level infrastructure first. That is one of the reasons we have switched our projects over to DragonFly. I am tired of systems that are like a sky scraper built on top of a rickety foundation. That is one of the reasons Linux became so unstable that we left it after being on it for at least 10 years, and again with FreeBSD. When FreeBSD-5.x became so unstable that we could not even complete NFS backups without the server hanging, and stayed that way for months, we left it and went to NetBSD, which is still a pretty nice system, although not as feature rich in a lot of ways. We did not wait for years for FreeBSD to stabilize like we did with Linux (Hard lesson learned) before discovering it was not going to. Of course, I hope that does not prove true with FreeBSD. FreeBSD still seem to have a more professional development community than Linux. However, we have since switched to DragonFly because of architecture, planning, developer attitude, and emphasis on clean code and stability. I believe a perfected low level infrastructure should pay off in the long run with accelerated development without loosing stability. I have been impressed with the progress of DragonFly so far, especially considering the comparatively modest number of developers. Vince
Re: laptop lcd vs external lcd panel Xorg
: I installed 1.12.0 on my laptop along with 1.10.1 packages (should the : packages really be recompiled or can I use 1.10.1 binary packages?). I : :I don't have a good answer on the xorg difference between releases, but: :most/all of the 1.10.1 packages should work, as I understand it. We'll :have 1.12 binary packages soon; I have to get through a build on pkgbox. Yah, I noticed those were 1.10 built binaries, but I decided not to let it hold up the release. My only worry would be with the libc adjustments to the directory scanning code and the threading library switch. -Matt Matthew Dillon [EMAIL PROTECTED]
Re: FreeBSD 7, DragonFly's status
Stability is important to me but I also recognize that even the best project can become stale if one does not choose to develop the right aspects of it. A weakness in DragonFly is that it took a while to get to the more interesting things, like HAMMER, and will take yet longer to get to SSI. One of FreeBSD's strengths is that its brute-force method of development tends to pull in more interest simply by the sheer number of projects being worked on parallel. One of its weaknesses is a lack of stability. I far prefer our low level infrastructure. Our abstractions are an order of magnitude cleaner: VFS, timers, schedulers, cpu messaging, threading, namecache, VM paths, virtualization, PRNG, network drivers, network protocols, route table, and the list goes on. I am also very happy that all that infrastructure work is now basicaly done and I can focus on the more interesting aspects of the project. How do I explain to a lay person why moving the responsibility for namespace and I/O atomicy (range locking) into the kernel was important? It's hard. -Matt
Re: FreeBSD 7, DragonFly's status
On Fri, Feb 29, 2008 at 5:05 AM, Matthew Dillon [EMAIL PROTECTED] wrote: Well, I'll give you my 5-second opinion. What I am not worried about: ... * Ports and packages. This was a huge worry of mine at the beginning of the project. I no longer worry about it. Yeah, pkgsrc is a real life saver. I discovered NetBSD around the same time DragonFly was picking up pkgsrc, so it was very pleasant for me to have a reasonably uniform administrative strategy across the two operating systems. What I am worried about: * The big-ticket items that we have traditionally compared ourselves against, SMP primarily, have developed slowly, primarily because up until recently I was the only person working on it. It an issue of man-hours more then anything else. FreeBSD simply has more developers working on SMP then we do. We have the infrastructure and the abstraction, we now need to get down to the brass balls and finish it. Well yeah, that's my point. FreeBSD's vast developer base lets it get away with relatively unclean code and still remain relevant in performance and functionality. Stability has picked up a lot too, although that's hard to measure with hard evidence. * I really want to see someone take up the ball on getting the network stack MP safe. It's important to the project but I can't do that and HAMMER at the same time. It's the easiest thing to make MP safe in the project. Wasn't it almost done years ago, and then virtually untouched thereafter? I haven't been keeping up with source-level changes like that, but I got the impression the last 10% takes 99.99% of the time which is why it's still not done. * Similarly with AMD64. We need it. I've developed the infrastructure separation required and we even have a fully virtualized kernel (vkernel) which demonstratres the infrastructure separation. Most of the generic kernel code can easily support a 64 bit architecture. The compiler supports it. It's a project waiting for someone to take up the ball. * Our interrupt routing subsystem really needs a major upgrade. (i.e. a major port from FreeBSD). That's pretty much a showstopper for a lot of deployments. It's things like that which I believe bring the project down. Even if it's only a problem for 6 months, that's 6 months worth of new users which may be discouraged from using the system, or use another system temporarily that never gets replaced with DragonFly again. This has brought down Linux and other projects a lot in the past, where a critical bug was left unchecked long enough that hundreds or thousands of users took years to try again. I contend that fixing major problems must come before implementing new technology, especially if that new technology is likely to add new problems. If somebody installs DragonFly just to try HAMMERFS and discovers half of his hardware fails because of an interrupt bug, what good does that do? Where I think the future is: * SMP, Storage, and SSI. Real time mirroring at the logical level. HAMMER is a major component for the storage, SSI, and mirroring components, and I believe HAMMER will be a large interest magnet. I for one don't like to speculate about the future like that, because such gambles often cost a lot even to companies who have a lot of information from which to predict. Although it's been said the best way to know the future is to invent it, a lot of inventors have had their efforts wasted because of the lack of marketability of their inventions. Our project goals have not changed, but if I had it all to do over again I would have started work on HAMMER much earlier then I did. I spent more time then I should have perfecting the low level infrastructure, trying to build a base upon which all the other work could occur. No, I agree with everyone who says the low level stuff is what makes this project great in the first place. I'm just concerned it's nowhere near enough to take enough market share that SSI becomes truly useful. If it's just 1000 hardcore geeks in the whole world split into a few SSIs, what practical benefit is there? They may as well SSH in to a few powerful machines, and even consumer machines are becoming ludicrously powerful lately. The security and efficiency of shell access is a much more mature field than the security and efficiency of SSI in general, saying nothing of any specific implementation. And then, application clustering for web serving, databases, etc. is already largely solved on the application level, often much more optimally than a general approach could ever be. SSI doesn't help there either, unfortunately. -Matt -- Dmitri Nikulin
