Re: Cache coherency, clustering, and Kernel virtualization

2006-09-04 Thread [EMAIL PROTECTED]
Hi! Joerg Sonnenberger wrote: Yes and no. Xen itself needs a host kernel for the hardware support. Current implementations are Linux and NetBSD for Xen3, FreeBSD support is in progress (IIRC). It is also port of Plan9... Opensolaris guys have their kernel running in domain0. I would love to h

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-04 Thread Anil Madhavapeddy
On 4 Sep 2006, at 09:36, Matthew Dillon wrote: Xen is the one that runs a linux kernel as the primary OS? I really have no desire to make DragonFly dependant on some other OS for features. And, frankly, I'm not sure I would consider Xen to be contemporary to an actual nativ

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-04 Thread Ivan Voras
Joerg Sonnenberger wrote: And why reinvent the wheel? Xen is providing exactly that -- light weight virtualisation similiar to what IBM provided in hardware. It has the big advantage of being proven technology and does support multiple kernel already. I can only say that Xen looks like a good

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-04 Thread Steven Smith
> :And why reinvent the wheel? Xen is providing exactly that -- light > :weight virtualisation similiar to what IBM provided in hardware. > :It has the big advantage of being proven technology and does support > :multiple kernel already. > : > :Performance numbers are available as well, depending o

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-04 Thread Joerg Sonnenberger
On Mon, Sep 04, 2006 at 01:36:27AM -0700, Matthew Dillon wrote: > Xen is the one that runs a linux kernel as the primary OS? I really > have no desire to make DragonFly dependant on some other OS for > features. Yes and no. Xen itself needs a host kernel for the hardware support. Cur

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-04 Thread Matthew Dillon
:On Sat, Sep 02, 2006 at 11:49:36AM -0700, Matthew Dillon wrote: :> Sounds kinda like what IBM did with linux on its mainframes, eh? But I :> am going to do it with DragonFly and my expectation is that performance :> within a virtual kernel will be within 20% of the performance of a :

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-04 Thread Joerg Sonnenberger
On Sat, Sep 02, 2006 at 11:49:36AM -0700, Matthew Dillon wrote: > Sounds kinda like what IBM did with linux on its mainframes, eh? But I > am going to do it with DragonFly and my expectation is that performance > within a virtual kernel will be within 20% of the performance of a >

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-03 Thread Matthew Dillon
:with "multi-grid clustering" I meant joining multiple grids with one :logical machine, basically what you're trying to achieve for december; :MPP = Massively Parallel Processing, a shared memory machine, which is a :cluster but not in the sense that several individual hosts work on a :problem

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-03 Thread Thomas E. Spanjaard
Matthew Dillon wrote: You will have to define the terms "multi-grid clustering" and "MPP host". with "multi-grid clustering" I meant joining multiple grids with one logical machine, basically what you're trying to achieve for december; MPP = Massively Parallel Processing, a shared mem

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-03 Thread Matthew Dillon
:Which is to be expected ofcourse, when clustering the real kernel. I :assume this would also bypass some of the code that is necessary for :multi-grid clustering? And that a real-kernel-cluster could act as a MPP :host for multi-grid clustering over the internet, as you plan? In that :case, I

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-03 Thread Thomas E. Spanjaard
Matthew Dillon wrote: :I'm not fully getting this I'm afraid. So, probably a stupid question: :In the end it will be possible to cluster the real kernels, right ? Or :are you saying you will always use userland kernels to partition :resources and "donate" resources to a cluster ? :(The idea of us

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-03 Thread Matthew Dillon
:Hi, :I'm not fully getting this I'm afraid. So, probably a stupid question: :In the end it will be possible to cluster the real kernels, right ? Or :are you saying you will always use userland kernels to partition :resources and "donate" resources to a cluster ? :(The idea of userland kernels is

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-03 Thread Pieter Dumon
Hi, I'm not fully getting this I'm afraid. So, probably a stupid question: In the end it will be possible to cluster the real kernels, right ? Or are you saying you will always use userland kernels to partition resources and "donate" resources to a cluster ? (The idea of userland kernels is really

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Bill Hacker
Justin C. Sherrill wrote: On Sat, September 2, 2006 9:43 pm, Bill Hacker wrote: Hmm... take a look at the evolution and far from trouble-free history of, to name just one, z/MV. What's a z/MV? A casual Google search turns up nothing recognizable. Sorry. Historical artifact. .Irish Alz

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Matthew Dillon
: :So how does Userland VFS and ZFS go into this? : :Petr Userland VFS needs to use SYSLINK. Making progress with SYSLINK requires a fairly quick development cycle, one that I am not achieving with the current kernel build/install/reboot cycle. It just takes too long, even with

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Matthew Dillon
:Virtualization serves as a crude but effective way to accomplish the :former, but not the latter. That a constrained subprogram should not be :allowed to adversely affect its container is important -- indeed it is :something that we should be able to take for granted these days. But to :use the ex

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Matthew Dillon
:Just out of curiousity, how do you intend to achieve this goal? My :uneducated :mind leans towards a solution where all page faults are trapped in the :actual :kernel and a lookup table is used to pass the faults upto the actual virtual : :kernel hosting the process. : :(I'm not contending it ca

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Eric Jacobs
On Sat, 2 Sep 2006 23:28:20 -0400 (EDT) "Justin C. Sherrill" <[EMAIL PROTECTED]> wrote: > On Sat, September 2, 2006 9:43 pm, Bill Hacker wrote: > > > Hmm... take a look at the evolution and far from trouble-free history of, > > to name just one, z/MV. > > What's a z/MV? A casual Google search t

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Petr Janda
So how does Userland VFS and ZFS go into this? Petr Matthew Dillon wrote: .. -Matt

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Justin C. Sherrill
On Sat, September 2, 2006 9:43 pm, Bill Hacker wrote: > Hmm... take a look at the evolution and far from trouble-free history of, > to name just one, z/MV. What's a z/MV? A casual Google search turns up nothing recognizable.

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Bill Hacker
Eric Jacobs wrote: On Sat, 2 Sep 2006 11:49:36 -0700 (PDT) Matthew Dillon <[EMAIL PROTECTED]> wrote: Now consider the problem of tying an entire kernel into an internet-based cluster. Does that sound like something that would be 'safe' to integrate into your real kernel? NO WAY! I

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Eric Jacobs
On Sat, 2 Sep 2006 11:49:36 -0700 (PDT) Matthew Dillon <[EMAIL PROTECTED]> wrote: > Now consider the problem of tying an entire kernel into an internet-based > cluster. Does that sound like something that would be 'safe' to > integrate into your real kernel? NO WAY! It is virtually

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Matthew Dillon
:Please excuse my newbness --- but how does this differ from UML? : :Thanks, :--TongKe I'm somewhat familiar with UML and from what I can tell they are fairly similar. I do not know about UML's scaleability, however, and insofar as I can tell UML is a standalone effort. What we are

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Matthew Dillon
:This is hardly the logical step forward beyond SMP, and does nothing to :make proper use of consumer-level NUMA equipment (AMDs Athlon64/Opteron :family of processors for example), I don't see multiple virtual kernels :work in a NUMA system. Or do you intend to not neglect this class of :machi

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Thomas E. Spanjaard
Bill Hacker wrote: Thomas E. Spanjaard wrote: This is hardly the logical step forward beyond SMP, and does nothing to make proper use of consumer-level NUMA equipment (AMDs Athlon64/Opteron family of processors for example), I don't see multiple virtual kernels work in a NUMA system. Or do you

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Bill Hacker
Thomas E. Spanjaard wrote: *snip* This is hardly the logical step forward beyond SMP, and does nothing to make proper use of consumer-level NUMA equipment (AMDs Athlon64/Opteron family of processors for example), I don't see multiple virtual kernels work in a NUMA system. Or do you intend to

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread TongKe Xue
Please excuse my newbness --- but how does this differ from UML? Thanks, --TongKe On Sat, Sep 02, 2006 at 11:49:36AM -0700, Matthew Dillon wrote: > As people may have noticed, I managed to get the first cut of the > cache coherency subsystem in place. Unfortunately, a great deal more >

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Bill Hacker
Matthew Dillon wrote: *snip* Sounds kinda like what IBM did with linux on its mainframes, eh? AIX5L + Linux? Earlier roots than that. Over 32 years, and still in use. Closer to IBM CRL's CP/67 --> VM -- -- -- z/OS's z/VM *snip* I finally figured it out, and the answer is so sim

Re: Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Thomas E. Spanjaard
Matthew Dillon wrote: Consider what we want to accomplish. We want to be able to cut up system resources and link them into 'clusters', with the whole mess tied together on the internet. Originally I envisioned cutting up memory, disk, and cpu resources and connecting them to a

Cache coherency, clustering, and Kernel virtualization

2006-09-02 Thread Matthew Dillon
As people may have noticed, I managed to get the first cut of the cache coherency subsystem in place. Unfortunately, a great deal more work is needed to get it working fully. I need to flesh out syslink and work on the cross-machine cache coherency algorithms themselves. This