On 2/3/07, Petr Janda <[EMAIL PROTECTED]> wrote:
From what I gathered across this mailing list. DF can't scale nowhere close to Linux at the moment because DF still operates under the BGL.
And we won't know how well it scales until it supports the whole architecture necessary for those 4096-CPU systems, then find somebody who's willing and able to administer and fairly benchmark the system. It's not even enough to optimize the scaling. Lock contention is also a factor of the raw performance - more attempts at locking mean a higher rate of lock contention, but then if those locks run for less time there's less chance of an actual block, and indeed less harm when the block occurs. You still get the necessary instructions (and bus lock cycles, the real killers) to do the locking, but the emergent effect is different. Really short critical paths can use spinlocks sanely, and it's not clear what can be locked by a spinlock until it's optimized enough. Even DragonFly's parallel architecture requires a sub-architecture of fine grained locking. What I'm saying is, even if the BGL disappeared overnight, there'd be a lot more optimizing and profiling left to make DragonFly bench against Linux without humiliation. And Linux has years of head start with a much larger developer base, including countless corporate and governmental interests, plus very good technologies only it can use legally because of the patent system which pretty much guarantee its victory. The best DragonFly can do is be so much better at SSI clustering that Linux' high-end SMP performance is considered an expensive alternative, rather than actually viable. That means also beating Linux' years of clustering development. It's probably not too far off that if a corporation decided Linux' clustering needed improving in a way similar to DragonFly's design, it would be hacked in over a few months and would perform faster than DragonFly would for years just because of the foundation. It doesn't matter to them if it's maintainable or beautiful code, because on average for these projects the extra maintenance work is less than the work to make the code easy to maintain. And Linux' components, from the VM to the ATA to firewire to schedulers, are so often re-developed that they often have multiple alternatives for each system in the tree at any one time, with easy compile time switching. Even if nobody wants to maintain a crappy part of the system, somebody will just write an alternative and it's generally better. This haphazard and decidedly chaotic strategy maps so well to actual real-world development that it is the obvious underpinning of Linux' success. Any look at the code will make it abundantly clear that it's not software quality that gets it places, but being so welcoming to actual development by basement hackers and corporations alike. And that's something I'm not confident any other project can repeat now. Call me pessimistic... but I'd rather be optimistic about Linux' quality improving, than hoping for another project to replace a significant share of it. --- Dmitri Nikulin Centre for Synchrotron Science Monash University Victoria 3800, Australia
