ANNOUNCE: Linux Kernel ORB: kORBit
This email is here to announce the availability of a port of ORBit (the GNOME ORB) to the Linux kernel. This ORB, named kORBit, is available from our sourceforge web site (http://korbit.sourceforge.net/). A kernel ORB allows you to write kernel extensions in CORBA and have the kernel call into them, or to call into the kernel through CORBA. This opens the door to a wide range of experiments/hacks: * We can now write device drivers in perl, and let them run on the iMAC across the hall from you. :) * Through the use of a LD_PRELOAD'd syscall wrapper library, you can forward system calls through CORBA to an arbitrary local/remote machine. * CORBA servers are implemented as Linux kernel modules, so they may be dynamically loaded or unloaded from a running system at any time. CORBA servers expose their IOR's through a /proc/corba filesystem. * Filesystems may be implemented as remote CORBA objects and mounted on the local machine, by using 'mount -t corbafs -o IOR:... none /mnt/corba' This are just some of the features available _RIGHT_NOW_ that are supported by kORBit. I'm sure that YOU can think of many more. Implementation: We implemented this port by providing a user-kernel mapping layer that consists of providing standard system header files for the "user" code to #include. In these header files, we do the mapping required. For example, we implement a stdio.h that #defines printf to printk (as a trivial example). Only user level code sees or uses these wrappers... all of our modifications to the Linux kernel are contained within the linux/net/korbit subdirectory. This is currently implemented with a 2.4.0test10 kernel, although forward porting should be very easy. This project was implemented as a cs423 semester project by Chris Lattner, Fredrik Vraalsen, Andy Reitz, and Keith Wessel at the University of Illinois @ Urbana Champaign. Unresolved issues: * Our poll model is not optimial. Currently we actually do a real poll on a (struct socket *) set. This causes relatively high latencies (on the order 1 second, worst case) for CORBA requests. Our waitqueues are not working quite as well as they should. :) * Security is completely unimplemented. Someone could use corba interfaces to read any file on your system, for example (if the CORBA-FileServer module is installed). Thus, this is really more for prototyping and development than actual real world use. :) If you have any questions or comments, please feel free to contact us at: Chris Lattner, Fredrik Vraalsen, Andy Reitz, Keith Wessel [EMAIL PROTECTED] btw, yes we are quite crazy, but what good is it to be normal and conformist afterall? :) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Sat, 9 Dec 2000, Mohammad A. Haque wrote: It was just an example. Basically, you'd be able to do in with just about any language that has ORBit bindings. Ben Ford wrote: Why would you *ever* want to write a device driver in perl??? Precisely... but also, there could be a case where perl would make sense. Consider an FTP filesystem. There performance is not dictated by the speed of the language, it's limited by bandwidth. It could make sense to write your almighty FTPfs like this: 1. Prototype it in Perl, get all the bugs out. 2. Rewrite in C in userspace, get all the bugs out. 3. recompile/relink in kernel space with no source modifications 4. ship product. :) The great thing that kORBit buys you is insulation of kernel space from the drivers that are running... kinda like a microkernel. I'm not going to start a flamewar here about Linux and microkernels, but when doing initial development work for a driver, the test/crash/reboot/fsck cycle is a real pain (okay, maybe journalling helps a little, but you get the idea). What we're offering goes more like this: 1. Boot kernel 2. Install corbafs module for example 3. Start test filesystem in user space 4. mount test user space filesystem 5. test it, oh crap, it segfaulted. 6. CorbaFS gets exceptions trying to communicate to server, which it relays to the kernel as -errno conditions. 7. You safely unmount corbafs 8. fix your bug 9. goto step #2. Which is arguably nicer. :) The whole idea is that a bastard driver shouldn't take your kernel down if you know it to be unreliable... if you trust the driver, then by all means, don't use kORBit. Also, kORBit is useful when you don't WANT something in the kernel... and I won't bring up the whole user level filesystem debate again... :) -Chris http://www.nondot.org/~sabre/os/ http://korbit.sourceforge.net/ -- = Mohammad A. Haque http://www.haque.net/ [EMAIL PROTECTED] "Alcohol and calculus don't mix. Project Lead Don't drink and derive." --Unknown http://wm.themes.org/ [EMAIL PROTECTED] = - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Don't worry about kORBit. Like most open source projects, it will simply die out after a while, because people don't find it interesting and there is really no place for it. If it becomes useful, mature, and refined, however, it could be a very powerful tool for a large class of problems (like moving code OUT of the kernel). I do have one sensible question. Given that corba is while flexible a relatively expensive encoding system, wouldn't it be better to keep corba out of kernel space and talk something which is a simple and cleaner encoding Very good point. I think this distills down to common misconception of CORBA: that it must be slow. Actually, a LOT of research has been done to improve CORBA performance, but you have to keep in mind one very IMPORTANT THING: CORBA does NOT dictate a transport or data format for the data going out "over the wire". CORBA does specify one "standard" used for interoperability: GIOP. It also specifies one important incarnation of GIOP: IIOP (GIOP over IP). kORBit currently uses this to communicate from user space to kernel space... obviously this is not ideal. :) There is absolutely NOTHING that prevents us from defining our own "data marshalling" protocol/algorithm that is lighter weight than IIOP. In fact, we had planned to implement that for our project, but of course, ran out of time. It would be relatively straight forward to define this extension in the ORBit code base, and in fact "ORBit 2" is striving to make this much much easier than it is now (basically sorting out IIOP specific code from the main code base better, and generally cleaning things up). Currently, ORBit supports IIOP and its own proprietary (as far as I know) Unix domain socket protocol (obviously only works intramachine). There is a large perception of CORBA being slow, but for the most part it is unjustified. I believe that the act of _designing_ a completely new protocol, standardizing it, and making it actually work would be a huge process that would basically reinvent CORBA (obviously some of the design decisions could be made differently, but all the same issues would have be dealt with). CORBA, today, gives us superior interoperability (through IIOP), with extensibility for the future. As Alexander Viro mentions, 9P may be a better protocol for local communications... so CORBA gives us a framework to try that out with. Even IIOP is smart enough to, for example, only do endian swapping of data if the two machines actually differ... so at least it doesn't specify a "network" byte order. CORBA isn't ideal for all applications, it certainly isn't the ideal IPC or RPC mechanism (the L4 designers would have a laughing fit if you tried to use it for _general_ IPC), but it is very useful for a lot of things. kORBit is very pre-alpha code, it works, but has lots of room for enhancements, experiementation, and toying around with. That's what makes it cool. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Err... how about this: Give me two or three kORBit syscalls and I can get rid of all the other 100+ syscalls! :) Like it ioctl() does it? Number of entry points is _not_ an issue. Diversity of the API is. Technically, kernel has 1 (_o_n_e_) entry point as far as userland is concerned. int 0x80 on x86. Can't beat that, can you? Err shame on you, don't forget about lcall and exceptions, and interrupts, and... That is technically more than _o_n_e_ "entry point". :) Oh wait, what about sysenter/exit too? :) No I can't beat that. But if you look at the hack job of a system call table we have, you can see that there is no _really_ standard way of passing parameters. Oh sure, most of the time, stuff is passed in registers. Sometimes we get a pointer to an argument struct. Because of this wonderful design we get all kinds of stuff like sys_oldumount vs sys_umount and others... Yes, standard RPC mechanism would be nice. No, CORBA is not a good candidate - too baroque and actually known to lead to extremely tasteless APIs being implemented over it. Yes, I mean GNOME. So sue me. Hrm... because I'm stupid, please explain how CORBA is too baroque... I have no problem with you not liking GNOME... you're a kernel hacker, so you're not supposed to like GUI's. :) [just kidding!!!] CORBA doesn't preclude nasty APIs any more than C does. It also doesn't preclude *nice* APIs that are upgradable and extensible in the future (and that means without breaking backwards compatibility). Please don't tell me that OOP is bad... or else we will have the eviscerate the VFS layer from the kernel (amount other subsystems)... :) I would take 9P over that any day, thank you very much. Like I mentioned in a previous email, CORBA does not preclude 9P. What it does buy you though, is compatibility with LOTS of preexisting CORBA tools. How much development infrastructure is there for 9P? I thought so. :) For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
I do have one sensible question. Given that corba is while flexible a relatively expensive encoding system, wouldn't it be better to keep corba out of kernel space and talk something which is a simple and cleaner encoding p9fs exists. I didn't see these patches since August, but probably I can poke Roman into porting it to the current tree. 9P is quite simple and unlike CORBA it had been designed for taking kernel stuff to userland. Besides, authors definitely understand UNIX... One thing that you might want to mention Alexander: 9P is not a general communications protocol. In fact, it doesn't work very well across the internet at all. To get decent performance, the Plan9 group (which, is a very cool group. :) has to specify a new protocol that competes with TCP on the level of complexity (IL: http://plan9.bell-labs.com/sys/doc/il/il.html) Also, 9P is a general communications framework only in the context of Plan9 itself. In reality it only applys directly/well to filesystem related issues... the reason it works well in Plan9 is that _everything_ is a file (part of the beauty of plan9). With some elbow grease, 9P could probably be made to work in the kORBit framework. It's not even that big of a deal: it just takes time. Believe me when I say that IIOP is not a very good user-kernel communications mechanism. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
/me trims down CC list... Local? Funny. It lives atop of TCP or IL quite fine. What's even funnier, I can use it to export /proc from CPU server to workstation and use _that_ for remote debugging. Ditto for window system. Ditto for DNS. Ditto for plumber. No, not on Linux... No not on linux. No definately not on Windows, no not on the BSD's either. Oops, wasn't interoperability an important part of the Linux kernel design? Didn't we want to use and follow and define _real_ standards? What you do is mapping your RPC to hierarchical namespace. After that... What extensibility do you need? You can use every utility written for core UNIX API (open()/read()/write()/close()) as a client. No need to reinvent the wheel... Heh, that's interesting. _getting_ the data isn't hard in 9P, actually using it or displaying it in a meaningful way commonly requires something more than cat. :) The overriding problem with a simplistic 9P port is that linux kernel functionality is not very well modularized. In fact, this prevents wonderful CORBA integration as well: An agressive filesystem (for example) often likes to touch the buffer cache or even the VM layer (*cough*) directly. Some don't use the prescribed interfaces, because nothing enforces it. 9P works wonderfully for Plan9 because they had the luxery of redefining/rewriting the whole OS, and the whole interaction with user space processes. We, unfortunately, don't have that possibility. :) Please don't get me wrong. I'm not opposed to other ideas, and 9P may in fact turn out to be a very nice protocol that would be able to support much kORBit level functionality). I do maintain that by writing a custom user-kernel marshalling library, you could obtain better peformance than 9P though, because you could take advantage of lots of machine specific optimizations. Hell you could even pass things in register if you'd have = 4 args. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
either. Oops, wasn't interoperability an important part of the Linux kernel design? Didn't we want to use and follow and define _real_ standards? Erm... 9P stub exists for Linux. It exists for FreeBSD. I suspect that it exists for other *BSD too - never checked that. Okay, so there are _stubs_ for these platforms. How many languages are there bindings for? What about Win32 (gross I know, but at least for now, it is important)? What about OS/2 what about Amiga and palm? CORBA is very popular for certain things... even inferno isn't nearly as big... Heh, that's interesting. _getting_ the data isn't hard in 9P, actually using it or displaying it in a meaningful way commonly requires something more than cat. :) Same for any other RPC mechanism. Of course. Which is why CORBA is about putting STRUCTURE in that stream of random bytes coming over the wire. Why should I have to rewrite my marshalling and demarshalling code every time I want to write a server. read and write are fine. But sometimes I want a structure. Sometimes, my structures aren't laid out like C struct's either. What then? What if I want to send an "object" to you? nothing enforces it. 9P works wonderfully for Plan9 because they had the luxery of redefining/rewriting the whole OS, and the whole interaction with user space processes. We, unfortunately, don't have that possibility. :) Notice that they _don't_ export random internal APIs to userland. And neither do we. Beautiful isn't it? :) Please don't get me wrong. I'm not opposed to other ideas, and 9P may in fact turn out to be a very nice protocol that would be able to support much kORBit level functionality). I do maintain that by writing a custom user-kernel marshalling library, you could obtain better peformance than 9P though, because you could take advantage of lots of machine specific optimizations. Hell you could even pass things in register if you'd have = 4 args. :) You mentioned OOP, didn't you? Encapsulation is a good thing and what you are talking about is "layering violations made Real Easy(tm)". I simply don't see why _that_ is a good goal. I completely fail to see how I'm breaking encapsulation or making layering violations easy. Why I'm talking about is the fact that CORBA doesn't care how you ship bytes around: it's a higher level protocol that cares about shipping "things" around. Bytes are old school, "stuff" is the wave of the future. ;) heh. -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
plan-9.bell-labs.com/sys/man/ Arrgh. s/plan-9/plan9/. My apologies. Cool, thanks, will read. :) IDGI. What 9P gives is an RPC mechanism that uses normal (as in "named streams of characters") representation on the client side and very light-weight library on the server side. It looks like you are trying to do a mechanism that would export arbitrary _internal_ kernel APIs. I really don't see what you would want it for. Which is exactly why it doesn't work well for many applications. The problem is this: how do you get from a byte stream to a structured data stream? There are many answers: 1. Keep your data structures so simple, that it's obvious. Not a good choice. :) 2. Defined interfaces to the bytestream for every interface that you define. 3. Define a standard for inflating/deflating "things" into bytestreams. Oh wait, that's what corba does. :) The point of the matter is that CORBA gives you a nice, clean, well supported method of not having to write marshalling/demarshalling code every time you want to use an interface. Not only that, it lets you define "fast paths" for special cases (ie, with user level corba, if you are communicating between two objects in the same processes, you can do direct calls and do no marshalling). These fast paths do not interfere with the generality of the system at all. The 9P way of doing things is not fundementally new, they just applied the idea that "everything is a file" more broadly. What annoys me is that it is not immediately obvious how to "demarshall" the data that comes out of /dev/mouse for example. Combine that with the problem that /dev/mouse might change format in the future (okay stupid example, but you get the idea) to use floating point coordinates, and things certainly get hairy. Why reinvent the wheel countless times? -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: CORBA vs 9P
Okay, so there are _stubs_ for these platforms. How many languages are there bindings for? Grr... Let's define the terms, OK? What is available: kernel code that represents the client side of RPC as a filesystem. Userland clients do not know (or care) about the mechanisms involved. But they DO CARE about the format of the data. And files with structure are things of dreadful past. BTDT. You really need to... work with an OS that would have and enforce "structured files" spit to appreciate the beauty of ASCII streams. Ahhh, so ASCII streams are a wonderful thing. Are you an XML fan? :) However, that's a different story. What I _really_ don't understand is the need to export anything structured from kernel to userland. Okay, how about a few examples. How about /proc/meminfo? How about the "stat" structure? How about /proc/stat? You seem to be indicating that ASCII files are fine for general exportation of kernel information. The /proc filesystem begs to differ. One specific example is the /proc/meminfo file. Why is it that one field is 0 now? Ouch we can't change the format of the file because we'll break some program. Crap, you want to add a field, well, tough luck. The struct stat example is one _trivial_ example of "the need to export anything structured from kernel to userland". IOW, I would really like to see a description of use of your mechanism. If it's something along the lines of "let's take a network card driver, implement it in userland and preserve the current API" - see the comment about layering violations. You've taken an internal API and exposed it to userland in all gory details. See also your own comment about internal APIs being not convenient for such operations. I'm not trying to dictate interfaces. I'm not trying to tell people what to use this stuff for. I'm arguing that it's useful and that you can do very interesting things with it. If it's something else - I wonder what kind of objects you are talking about and why opaque stuff (== file descriptors) would not be sufficient. Opaque stuff is fine. I have no problem with file descriptors. They effectively solve the exact same class of problems that CORBA does, except that they add significant _API BLOAT_ because every little "method" that implements them gets a syscall. Oh yeah, they you get ioctl too. Funness. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
OK, now I'm completely confused. * which complex data structures do you want to export from the kernel in non-opaque way? * which of those structures are guaranteed to remain unchanged? * if you have userland-to-userland RPC in mind - why put anything marshalling-related into the kernel? Okay, I think I did my best to completely confuse you. :) #1: CORBA objects _are_ opaque. #2 is irrelevant due to #1. #3: userland-userland is not the interesting part. We want kernel-userland or user-kernel, as the common (ie fast) case, but we also want to do client-client or kernel-kernel without anything breaking. /dev/mouse for example. Combine that with the problem that /dev/mouse might change format in the future (okay stupid example, but you get the idea) to use floating point coordinates, and things certainly get hairy. HUH? OK, suppose it had happened. Do you really expect that you will not need to change your applications? I mean, if you expected a bunch of YES. You should not have to change your apps at all. ints and received a bunch of doubles... You either need to decide on rounding (and it's a non-obvious question) or you need to change quite a bit of code in your program. It goes way past the demarshalling, no matter whether you use CORBA, 9P or printf/scanf. NO. You want leagacy program to "just get" rounded ints, and new programs to get the "full precision" of the floating point #'s. OK, suppose you have a CORBA-based system and mouse drivers' API had been changed - they really want to return floating point coordinates. How will CORBA help you? Aside of making your programs scream aloud, that is. Okay, maybe this isn't a terrible example afterall. :) Consider this theoretical API for a mouse, that I'm certainly not saying is wonderful or perfect, just consider it to be an example (on the fly), and no it doesn't let you poll for mouse activity, this is just off the cuff: Original interface for the mouse: interface mouse { void getPosition(out long X, out long Y); }; So if you get the mouse interface in CORBA (which is an opaque object), you get back something that you can make requests to, and, for example, get the coordinate the mouse is currently at. Okay, that's fine, 15 years later and after much legacy software has been developed, company X develops a new high precions floating point mouse. Well crap, don't want to change the interface. Now instead of getting that, you get the "mouse2" or "floaty mouse" interface (for sake of argument suppose that we actually did want to use FP arithmetic in the kernel, that's an artifact of a bad example, not a bad point :): interface floatymouse : extends mouse { void getFloatyPosition(out float X, out float Y); } Now people that get the "/dev/mouse" interface get a floatymouse. Does this break all that leagacy mouse wielding code? No, because we used inheritance, the floatymouse is a superset of the mouse, and the original interface still works. The server/kernel side of the floaty mouse just chops off the decimal places when you use the getPosition method. This is the kind of stuff that I'm talking about. The concept that a mouse is more than a byte stream... that there is actual structure that must be understood before a datastream can be used. Yes, I am very aware that there are other ways of doing this, but CORBA is a very general powerful system that is also quite mature (unlike kORBit. ;) I hope this helps clarify what the heck I'm talking about. :) -Chris btw, just imagine how much cleaner the mount interface could be... ;) http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
NO. You want leagacy program to "just get" rounded ints, and new programs to get the "full precision" of the floating point #'s. What rounded ints? Rounded to zero? To nearest integer? To plus or minus infinity? Does program have something to say here? The exact same thing that older mouse drivers did. In this particular case, it doesn't matter anyways, because it's a simple example. If you REALLY care about rounding those coordinates a certain way, it's trivial to add an extra component to the interface. ... My mouse example ... Oh, great. So we don't have to care about formatting changes. We just have to care about the data changes. IOW, we are shielded from the results of changes that should never happen in the first place. And the benefit being...? What the hell are you talking about? Did you even read my example? I was giving an example of extending an API, adding new functionality to it. Notice that we could equally well add /dev/floatymouse and everything that worked with old API would keep working. The only programs that would need to know about floats would be ones that would... need to know about floats since they want to work with them. Wonderful solution. This is exactly what you would _have_ to do. Are you aware of why devfs came into existence? It is because there are already so damn many device nodes in /dev. Lets keep adding more. Why is it (for example) that on my machine (mandrake 7 install, no devfs), I automatically get inodes for ISDN PPP!?! Well, because ISDN has a slightly different interface, and you're talking to a different driver that is not transparently hidden to the user. Why can't I just open /dev/net0 and get the first network device? Because we have so many inconsistent, poorly design, inextensible interfaces laying around, thats why. Notice also that I can say ls /dev/*mouse* and get some idea of the files there. I can't do that for your interfaces. And you know what? I can't do ls /dev/*net* and get all the network devices either. Actually, one of the very cool things about CORBA is that you can BROWSE/LIST/SEARCH all objects currently instantiated. Being able to browse all in kernel objects would be very cool. The point being: if your program spends efforts on marshalling it would better _do_ something with the obtained data. And then we are back to the square 1. Uh huh, go ahead and read the example I sent to you. Returning to your example, I could not tee(1) the stream into file for later analysis. Not unless I write a special-case program for intercepting that stuff. I don't see why it is a good thing. On the contrary, it would be pretty easy to do something like that with CORBA. No you wouldn't be able to use tee, but why would you want to tee a binary data file? The only reason that tee works in this situation is because it's agnostic to the format of the data coming off the line... your analysis program would have to have special purpose code to parse the file. EVERY consumer of "mouse data" would have to parse the file. That seems pretty silly to me. I also don't see where the need in new system calls (or ioctls, same shit) comes from. Notice that your way is much closer to new system call than read()/write() of the right stuff. A new syscall was one example. It would be very simple to implement this by making _yet_another_ device node in /dev and issue reads and writes to it. That's more of a syntactic issue than a symantic one. As for the proc/meminfo... What would you do to a userland programmer who had defined a structure like that? Let's see: way too large, ugly as hell, many fields are almost guaranteed to become meaningless at later point... It was not designed, it got accreted. And that's very mild description - judging by results it might as well be s/ac/ex/. There is no provision in CORBA that prevents the creation of ugly interfaces. There is no provision in C that prevents them either, as witnessed by this example. At least with CORBA, I can extend the interface to actually return meaningful results while still maintaining backwards compatibility with programs that don't actually look at that field (as is done now by sticking the 0 in there). The trick is having enough foresight to design robust interfaces. I'm not saying that I'm an expert at that, and obviously nobody can be. What I _am_ saying is that CORBA/kORBit gives you flexibility in how you do interfaces, and it lets you "future proof" them (obviously only within reason). Adding a new device node everytime an API gets changed is like defining a different API for every hard disk that exists. Seems kinda silly. -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
1. kORBit adds about 150k of code to the 2.4t10 kernel. 2. kNFS adds about 100k of code to the 2.4t10 kernel. So can you implement a kNFS server in kORBit that takes less than 50kB of RAM? Otherwise it's still a contributor to bloat and this argument won't work ;) Actually the kORBitNFS server would have to take -50K of code to break even. :) The point was that kORBit lets you do a lot more... so hopefully that 50k of generality gives you something. :) I guess it's time to stop the flaming and to see what can be achieved using kORBit. The people who favour kORBit should IMHO be left alone and given the opportunity to show what can be achieved with kORBit ... if they don't achieve anything, the nay-sayers can always claim their "victory"; if something useful comes out the kORBit people can claim usefulness. Agreed! -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ORBit speed measure
There is a large perception of CORBA being slow, but for the most part it is unjustified. Well, I've measured using function calls through ORBit is 300 times slower than using dynamic loading. ... Which gives me the dl is about 333 times faster than ORBit. You leave so many details out, and you sortof miss the point with my statement. The idea is that CORBA doesn't specify a transport to use _at all_ except when doing inter-orb communication (which you basically have to use IIOP to be compatible). This is why, for example, if you ior-decode ORBit IORs that they contain two parts: an IIOP portion, and a Unix domain socket portion. Basically ORBit is optimizing the intramachine case, because it is very common for purposes that ORBit is mainly used: GNOME. You didn't specify, but I assume that you were using the Unix sockets transport for your benchmark. I am also assuming that the two things communicating were in different processes. If you wanted to make a more fair test case, you would have to use some form of IPC mechanism to communicate between two distinct processes. Comparing a function call to an RPC mechanism will never look good. :) There are many other optimizations that one can make the transport faster that ORBit doesn't implement. For example, you could mmap (shared) data buffers between the two processes communicating (of course, you still need to wake processes up, which is why it hasn't been done yet), or you could use pipes (if you can control forkness) or any number of things. If you are communicating with a server in the same process as your client (because it was dynamically loaded) you can even do a direct function call. When I said that "for the most part it is unjustified" to think that CORBA is slow, what I meant is that CORBA isn't slow: implementations are. Just look at the difference between ORBit and all those other orbs out there... orders of magnitude speed differences, with little or no functionality differences. What CORBA gives you is the power to have optimizations for important special cases (in kORBit's case, one of those would be User-kernel), WITHOUT sacrificing the big picture or the ability to be interoperable... I am not claiming that kORBit is where it needs to be right now. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
that is not transparently hidden to the user. Why can't I just open /dev/net0 and get the first network device? Because we have so many inconsistent, poorly design, inextensible interfaces laying around, thats why. This is a bad example, but a (perhaps?) good point. It seems it should be possible to implement an unlimited number of TCP/IP devices, each Part of my argument is that although every kernel interface boils down to something as simple as ioctl (where you have lots of api's multiplexed onto one data sink, int 0x80 is another example, kORBit is another), some ways of doing this multiplexing are better than others. I look at it as a hierarchy: 1. CORBA/kORBit 2. int 80/lcall 3. ioctl Each level is more structured than those below it. Each interface is also "cleaner" than those below it. I don't think that anyone would argue that we should replace int 80 and friends with an interface in the spirit of ioctl (even though it would be functionally identical). representing a connection, and each connected, disconnected, etc. by ioctls... Of course, I could be way wrong about this ioctl's... they come up again. Everytime that I look at the beautiful, clean, abstract interface that Unix exposes... it makes me happy. But unfortunately, if you look closer, you realize that the Unix API isn't really clean or nice at all... there is this dumping ground for odds and ends that don't fit into the standard model. So when you show someone the standard model, it looks clean and pretty... but someone always forgets to mention ioctl. I would claim that ioctl is one of the biggest reasons that kORBit needs to exist. See below. I can't do ls /dev/*net* and get all the network devices either. Actually, one of the very cool things about CORBA is that Different network devices are _very_ different... You can't broadcast on a modem, for instance, nor can you use promiscuous mode on a modem... You also can't do byte-by-byte raw mode on an ethernet... Precisely. But they do all have very common interfaces. For example, the standard network interface would probably have "transmit/receive block" and "getstatus" commands. This does not mean that they cannot implement other interfaces, however. Imagine this situation for writing your packet sniffer: 1. You open up /dev/net0 and get the Network interface 2. You query the network interface to see if it has an implementation of the promiscuous interface. 3. If not, you bail, because it doesn't support it. 4. If, you go ahead and use it. Contrast that to: 1. Survey all the different interfaces that are known (at development time of course) to have promiscuous interfaces. 2. Find out which ones are active. 3. Depending on what kind of interface it is, load a library that can understand the byte format comming off the device. 4. Use that library. The problems with the second (currently used) approach is that the APPLICATION writer has to keep up with new hardware developments and new interfaces. They have to understand and code parsers for the bytestreams coming from each device, which (as you mentioned) are all different, because they all support different (although overlapping) extensions. All of these calls that currently get dumped into ioctl (because they are not important enough to warrant an API function at the top level) suddenly become well structure and well designed interfaces. For example, if your device doesn't support the terminal handling ioctls the system has to basically figure that out and report errors on it. With CORBA/kORBit, you would just not implement that interface... so if someone asked for it, they would get a null pointer to the interface, which they check for and realize that it's impossible to do terminal stuff on. isatty (an example of an IOCTL wrapper) suddenly becomes implicit and the API is cleaner... I have no problem with the "unix way", but I do think that it can be augmented in some ways... -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
There is a large perception of CORBA being slow, but for the most part it is unjustified. I believe that the act of _designing_ a completely new CORBA is slow compared to some of the other solutions. The question I was trying to ask is whether you should put something smaller and faster into the kernel space and leave CORBA in userland. What I'm trying to show is that CORBA itself is not slow. CORBA can be thought of as an IDEA, and the current implementations are suboptimal. For example, no CORBA implementation has been tweaked to run well in-kernel. Because of this, people will say that CORBA sucks and is slow, but that's not true. I will agree that IIOP (the standardized corba communication mechanism) is much slower and more complex than we would want for a generalized user-kernel communication mechanism. The nice thing about CORBA is that you can design your own transport to optimize things. Thus 99% of the time you could use a heavily optimized, very simple transport that doesn't do much. This is what you are asking for, and it make perfect sense. What CORBA buys you is the ability to use this streamlined interface without giving up full generality... It's complex, it has security implications surely it belongs talking something simple and fast to the kernel. First off, there are a lot of complex systems in the kernel. :) Second, there are not security implications above and beyond any normal kernel hacking. The reason that kORBit has security implications right now is that there _IS NO SECURITY_. Imagine implementing sys_read with no checks of what uid is running, and you get the idea. Actually security in CORBA can be done BETTER in kernel space than in user space, but I digress... If you look at microkernels they talk a much simpler faster rpc protocol. Yes but for the most part they lose all of the advantages of CORBA as well. For example, every microkernel that I'm aware of can only talk to remote servers/clients that are running the same microkernel on the remote machine. There are no provisions for byte swapping if the endianness is incorrect or for having structured bytestreams (so that you can tell things about the raw bytes coming over the line without understanding them all). -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Of course. Which is why CORBA is about putting STRUCTURE in that stream of random bytes coming over the wire. Why should I have to rewrite my marshalling and demarshalling code every time I want to write a server. read and write are fine. But sometimes I want a structure. Sometimes, my structures aren't laid out like C struct's either. What then? What if I want to send an "object" to you? Then I need to understand the object anyway. And Corba objects are horribly over complex. Any lisp hacker will tell you there is only one type: a list. But alan, that's the beautiful thing. Given a CORBA object, you can understand its structure without knowing exactly what the contents are. You can effectively derive it's prototype just by inspecting it. The only difference between lisp and a CORBA object in this respect is that each item in the list is typed so that you have even more info about what to do with it. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
There is a large perception of CORBA being slow, but for the most part it is unjustified. Really? I have that same perception but I can't claim that I've measured it. On the other hand, I have measured the overhead of straight UDP, TCP, and Sun RPC ping/pong tests and you can find the code for that in any version of lmbench. It should be a 5 minute task for someone who groks corba to do the same thing using the same framework. If someone wants to do it, I'll guide them through the lmbench stuff. It's pretty trivial, start Urm... thanks for the offer... but you misunderstand me if you think that I'm claiming that kORBit is the ideal/fast implementation that everyone has been looking for. There is still much to be done. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Oh, great. So we don't have to care about formatting changes. We just have to care about the data changes. IOW, we are shielded from the results of changes that should never happen in the first place. And the benefit being...? What the hell are you talking about? Did you even read my example? I was giving an example of extending an API, adding new functionality to it. Yes, I did. What I don't understand is how kernel mechanism for marshalling would make your life easier wrt changes. I gave a very simple example of how an interface could be designed and then later extended without breaking any user space programs. Please reread the example and tell me precisely what doesn't make sense, and I would be glad to explain it better. Notice also that I can say ls /dev/*mouse* and get some idea of the files there. I can't do that for your interfaces. And you know what? I can't do ls /dev/*net* and get all the network devices either. Actually, one of the very cool things about CORBA is that you can BROWSE/LIST/SEARCH all objects currently instantiated. Being able to browse all in kernel objects would be very cool. So what's to stop you from letting readdir() do the work? WTF do we need one more API for returning the list of objects? Urm.. that was the point. You don't add a new API (it's built into corba, along with introspection). Also, like I JUST MENTIONED, readdir will have no way of knowing what is a network device in the quagmire of stuff floating around in /dev. (ls *net* does actually call readdir ya know. Perhaps you should read up more on unix... [sorry couldn't help it. ;]) The point being: if your program spends efforts on marshalling it would better _do_ something with the obtained data. And then we are back to the square 1. Uh huh, go ahead and read the example I sent to you. Did that. So they you understand that no interface was broken even though more data is now available... Returning to your example, I could not tee(1) the stream into file for later analysis. Not unless I write a special-case program for intercepting that stuff. I don't see why it is a good thing. On the contrary, it would be pretty easy to do something like that with CORBA. No you wouldn't be able to use tee, but why would you want to tee a binary data file? The only reason that tee works in this situation is For the same reasons why I use tar, gzip, whatever. I don't _want_ to invent a new utility every time when Joe Doe adds a new piece of interface. Data is data is data. I can uuencode it and send to somebody who would care to analize the bloody thing. Do you mean that I need to write a special tool for that? For _every_ member of every interface somebody decided to add? I don't think so. I really don't. That's fine, I don't blame you. Ya know what? I even agree with you (image that? hehe :). You seem to forget that a CORBA object is exactly the same thing: a stream of bytes. Actually it does better than that. It's a stream of bytes that you can introspect on and determine the structure of. Imaging this. You have a bytestream flowing out of some pipe/device/socket/file/whatever and you pipe it to a "Decode" program. The decode program spits this out: sequence struct { float float } This example is meant to illustrate that _in a general way_ you can decode the structure of the bytestream. (hence my claims about CORBA adding structure to a standard bytestream)... this is quite a bit more useful than trying to figure out what 'cat' is trying to tell you about binary data (especially if it's floating point numbers). line... your analysis program would have to have special purpose code to parse the file. EVERY consumer of "mouse data" would have to parse the Or I would look at the size. Or I would say od and look at the result. Don't tell me what to do with the data, when I'll need to parse it I will. And if you expect me to bother with writing more stuff when generic tools would work fine - too bad, I've been there, done that and I'm not coming back. Heh... I'm not trying to replace od or cat or tee or pipes. What I'm trying to do is give you a larger toolkit that gives you MORE power. Imagine you want to get the first float value (of each struct) of the above stream. Imagine a functional scripting language where you could do this (haskell syntax stolen gratitously): cat /dev/floatymouse | funcwith 'map fst' Which you could then point to the "decode" type program mentioned above, and you would now get: sequencefloat BTW, you cared about size of /dev, didn't you? /usr/bin choke-full of tee-foo, tee-bar, yodda, yodda would be better? You wouldn't have to replace the standard utilities. Sorry to rain on your parade. file. That seems pretty silly to me. Difference between good program and bad one: the former gets used in ways its authors never thought about... *cough* see above.
Re: ANNOUNCE: Linux Kernel ORB: kORBit
For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... Cool! However, can you do one test for me? Do _heavy_ writes on kORBit-ized box. That might show you some problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. I'll give that a try when I get a chance. :) I've always wanted to do this: redirect /dev/dsp from one machine to another. (Like, I have development machine and old 386. I want all programs on devel machine use soundcard from 386. Can you do that?) Yes. Definately. There are probably other ways of doing that... but one of the things we implemented was a "generic" character device... and we tested it by having a chardev server that basically reads from a "local" (to the server) character device, and forward it over CORBA. So this is already implemented! :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
cat /mnt/www/www.kernel.org/index.html can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came to conclusion that web filesystem is not possible... (If you can't do Yes, if the server supports webDAV or something similar. listings, it is not really filesystem; you could do cat /mnt/www/www.kernel.org_index.html as well, and that's easy to do.) and the CorbaFS userspace server takes care of loading the webpage and returning it to the kernel client. And these new filesystems don't take up any extra space in the kernel, since they all talk to the same CorbaFS kernel module! Not to mention being able to implement the filesystem in any language you like, debug the implementation in userspace, etc. codafs can do pretty much the same. Yes, but codaFS is specific to filesystems. kORBit, of course, can do much much more, in a very uniform way. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC] bloody mess with __attribute__() syntax
On Thu, 5 Jul 2007, Al Viro wrote: On Thu, Jul 05, 2007 at 09:41:55AM -0700, Linus Torvalds wrote: Note that gcc rules for __attribute__() (and that's the only source of rules we _have_ for the damn thing) clearly say that int __user *p; is the same thing as int *__user p; Quick question: is there some reason why we have to honor the crazy gcc rules, and cannot try to convince gcc people that they are insane? AFAICS, they started with storage-class-like attributes. Consider e.g. always_inline or section; these are not qualifiers at all and you want to have static __attribute__((always_inline)) int foo(int *p); interpreted with attribute applied to foo, not to its return type. This is true, but I don't think this is related. attributes in GCC can apply either to types or two decls. In this case, the always_inline attribute is being applied to the decl, but other attributes could be applied to the return type. -Chris -- http://nondot.org/sabre/ http://llvm.org/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
ANNOUNCE: Linux Kernel ORB: kORBit
This email is here to announce the availability of a port of ORBit (the GNOME ORB) to the Linux kernel. This ORB, named kORBit, is available from our sourceforge web site (http://korbit.sourceforge.net/). A kernel ORB allows you to write kernel extensions in CORBA and have the kernel call into them, or to call into the kernel through CORBA. This opens the door to a wide range of experiments/hacks: * We can now write device drivers in perl, and let them run on the iMAC across the hall from you. :) * Through the use of a LD_PRELOAD'd syscall wrapper library, you can forward system calls through CORBA to an arbitrary local/remote machine. * CORBA servers are implemented as Linux kernel modules, so they may be dynamically loaded or unloaded from a running system at any time. CORBA servers expose their IOR's through a /proc/corba filesystem. * Filesystems may be implemented as remote CORBA objects and mounted on the local machine, by using 'mount -t corbafs -o IOR:... none /mnt/corba' This are just some of the features available _RIGHT_NOW_ that are supported by kORBit. I'm sure that YOU can think of many more. Implementation: We implemented this port by providing a user->kernel mapping layer that consists of providing standard system header files for the "user" code to #include. In these header files, we do the mapping required. For example, we implement a that #defines printf to printk (as a trivial example). Only user level code sees or uses these wrappers... all of our modifications to the Linux kernel are contained within the linux/net/korbit subdirectory. This is currently implemented with a 2.4.0test10 kernel, although forward porting should be very easy. This project was implemented as a cs423 semester project by Chris Lattner, Fredrik Vraalsen, Andy Reitz, and Keith Wessel at the University of Illinois @ Urbana Champaign. Unresolved issues: * Our poll model is not optimial. Currently we actually do a real poll on a (struct socket *) set. This causes relatively high latencies (on the order 1 second, worst case) for CORBA requests. Our waitqueues are not working quite as well as they should. :) * Security is completely unimplemented. Someone could use corba interfaces to read any file on your system, for example (if the CORBA-FileServer module is installed). Thus, this is really more for prototyping and development than actual real world use. :) If you have any questions or comments, please feel free to contact us at: Chris Lattner, Fredrik Vraalsen, Andy Reitz, Keith Wessel <[EMAIL PROTECTED]> btw, yes we are quite crazy, but what good is it to be normal and conformist afterall? :) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Sat, 9 Dec 2000, Mohammad A. Haque wrote: > It was just an example. Basically, you'd be able to do in with just > about any language that has ORBit bindings. > > Ben Ford wrote: > > Why would you *ever* want to write a device driver in perl??? > Precisely... but also, there could be a case where perl would make sense. Consider an FTP filesystem. There performance is not dictated by the speed of the language, it's limited by bandwidth. It could make sense to write your almighty FTPfs like this: 1. Prototype it in Perl, get all the bugs out. 2. Rewrite in C in userspace, get all the bugs out. 3. recompile/relink in kernel space with no source modifications 4. ship product. :) The great thing that kORBit buys you is insulation of kernel space from the drivers that are running... kinda like a microkernel. I'm not going to start a flamewar here about Linux and microkernels, but when doing initial development work for a driver, the test/crash/reboot/fsck cycle is a real pain (okay, maybe journalling helps a little, but you get the idea). What we're offering goes more like this: 1. Boot kernel 2. Install corbafs module for example 3. Start test filesystem in user space 4. mount test user space filesystem 5. test it, oh crap, it segfaulted. 6. CorbaFS gets exceptions trying to communicate to server, which it relays to the kernel as -errno conditions. 7. You safely unmount corbafs 8. fix your bug 9. goto step #2. Which is arguably nicer. :) The whole idea is that a bastard driver shouldn't take your kernel down if you know it to be unreliable... if you trust the driver, then by all means, don't use kORBit. Also, kORBit is useful when you don't WANT something in the kernel... and I won't bring up the whole user level filesystem debate again... :) -Chris http://www.nondot.org/~sabre/os/ http://korbit.sourceforge.net/ > -- > > = > Mohammad A. Haque http://www.haque.net/ >[EMAIL PROTECTED] > > "Alcohol and calculus don't mix. Project Lead >Don't drink and derive." --Unknown http://wm.themes.org/ >[EMAIL PROTECTED] > = > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > It was just an example. Basically, you'd be able to do in with just > > > about any language that has ORBit bindings. > Agree. I remember a big complaint about Windows was the huge APIs, > compared with Unix' tiny list of syscalls. And then I saw the GNOME > docs... ew! Err... how about this: Give me two or three kORBit syscalls and I can get rid of all the other 100+ syscalls! :) We're not trying to port GNOME to the kernel. We're not trying to bloat the kernel API, actually we're trying to do the opposite. Consider this: 1. kORBit adds about 150k of code to the 2.4t10 kernel. 2. kNFS adds about 100k of code to the 2.4t10 kernel. 3. kORBit can do everything kNFS does, plus a WHOLE lot more: For example implement an NFS like server that uses SSL to send files and requests... so it is really actually "secure". > > Yeah... "Infinitely extendable API" and all such. Roughly translated > > as "we can't live without API bloat". Frankly, judging by the GNOME Whoa, you are so right... a well defined API is a terrible thing! Instead, lets try doing the following: 1. Define our API to be a bizarre and poorly documented collection of inline functions, macros, and normal functions. 2. Lets make this API have all kinds of subtle side effects, and lets not document these! 3. Lets raise the level that people have to be at to even TOUCH our APIs. Only uber-hackers that have worked with the system for 1+ years should be able to comprehend what's going on in the big picture... 4. Nobody that is "allowed" to work on this thing is dumb enough to produce any bugs. Because of that, debuggers are outlawed and anyone who attempts to debug code is shunned from existance... 5. Lets duplicate code all over the place that does mostly the same thing, but has subtle differences. 6. Lets structure the file layout such that files are in poorly grouped catagories, often with ambiguous locations, and lets drop HUNDREDS of .c files into individual directories. After all that's much easier than actually sorting them out. oh wait, we have this, the Linux kernel. Yes yes, this was definately a pot shot at the kernel tree, and I'm not at all serious about this. The fact of the matter, however, is that there are a lot of cool things that a well defined interface for kernel programming can buy you. Yes, there is an overhead, but do you really care about how performant your parrellel port scanner driver is? I'm not advocating that people write their fibre channel drivers with kORBit. :) Actually, if you really want to get technical about this thing, I never said anyone should use this at all. :) I just mentioned that it is out there and if people feel like it they can use it or play with it. I happen to think it addresses a point that is currently poorly addressed in the Linux kernel: you can't do RPCs to the kernel in a well defined manner. Linux is facing a crisis that is growing every day, and that is that there is no well defined interface for drivers to use... so compatibilty across releases it almost impossible. Don't worry about kORBit. Like most open source projects, it will simply die out after a while, because people don't find it interesting and there is really no place for it. If it becomes useful, mature, and refined, however, it could be a very powerful tool for a large class of problems (like moving code OUT of the kernel). -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > Don't worry about kORBit. Like most open source projects, it will simply > > die out after a while, because people don't find it interesting and there > > is really no place for it. If it becomes useful, mature, and refined, > > however, it could be a very powerful tool for a large class of problems > > (like moving code OUT of the kernel). > > I do have one sensible question. Given that corba is while flexible a > relatively expensive encoding system, wouldn't it be better to keep corba > out of kernel space and talk something which is a simple and cleaner encoding Very good point. I think this distills down to common misconception of CORBA: that it must be slow. Actually, a LOT of research has been done to improve CORBA performance, but you have to keep in mind one very IMPORTANT THING: CORBA does NOT dictate a transport or data format for the data going out "over the wire". CORBA does specify one "standard" used for interoperability: GIOP. It also specifies one important incarnation of GIOP: IIOP (GIOP over IP). kORBit currently uses this to communicate from user space to kernel space... obviously this is not ideal. :) There is absolutely NOTHING that prevents us from defining our own "data marshalling" protocol/algorithm that is lighter weight than IIOP. In fact, we had planned to implement that for our project, but of course, ran out of time. It would be relatively straight forward to define this extension in the ORBit code base, and in fact "ORBit 2" is striving to make this much much easier than it is now (basically sorting out IIOP specific code from the main code base better, and generally cleaning things up). Currently, ORBit supports IIOP and its own proprietary (as far as I know) Unix domain socket protocol (obviously only works intramachine). There is a large perception of CORBA being slow, but for the most part it is unjustified. I believe that the act of _designing_ a completely new protocol, standardizing it, and making it actually work would be a huge process that would basically reinvent CORBA (obviously some of the design decisions could be made differently, but all the same issues would have be dealt with). CORBA, today, gives us superior interoperability (through IIOP), with extensibility for the future. As Alexander Viro mentions, 9P may be a better protocol for local communications... so CORBA gives us a framework to try that out with. Even IIOP is smart enough to, for example, only do endian swapping of data if the two machines actually differ... so at least it doesn't specify a "network" byte order. CORBA isn't ideal for all applications, it certainly isn't the ideal IPC or RPC mechanism (the L4 designers would have a laughing fit if you tried to use it for _general_ IPC), but it is very useful for a lot of things. kORBit is very pre-alpha code, it works, but has lots of room for enhancements, experiementation, and toying around with. That's what makes it cool. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > Err... how about this: Give me two or three kORBit syscalls and I can get > > rid of all the other 100+ syscalls! :) > Like it ioctl() does it? Number of entry points is _not_ an issue. Diversity > of the API is. Technically, kernel has 1 (_o_n_e_) entry point as far as > userland is concerned. int 0x80 on x86. Can't beat that, can you? Err shame on you, don't forget about lcall and exceptions, and interrupts, and... That is technically more than _o_n_e_ "entry point". :) Oh wait, what about sysenter/exit too? :) No I can't beat that. But if you look at the hack job of a system call table we have, you can see that there is no _really_ standard way of passing parameters. Oh sure, most of the time, stuff is passed in registers. Sometimes we get a pointer to an argument struct. Because of this wonderful design we get all kinds of stuff like sys_oldumount vs sys_umount and others... > Yes, standard RPC mechanism would be nice. No, CORBA is not a good candidate - > too baroque and actually known to lead to extremely tasteless APIs being > implemented over it. Yes, I mean GNOME. So sue me. Hrm... because I'm stupid, please explain how CORBA is too baroque... I have no problem with you not liking GNOME... you're a kernel hacker, so you're not supposed to like GUI's. :) [just kidding!!!] CORBA doesn't preclude nasty APIs any more than C does. It also doesn't preclude *nice* APIs that are upgradable and extensible in the future (and that means without breaking backwards compatibility). Please don't tell me that OOP is bad... or else we will have the eviscerate the VFS layer from the kernel (amount other subsystems)... :) > I would take 9P over that any day, thank you very much. Like I mentioned in a previous email, CORBA does not preclude 9P. What it does buy you though, is compatibility with LOTS of preexisting CORBA tools. How much development infrastructure is there for 9P? I thought so. :) For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > I do have one sensible question. Given that corba is while flexible a > > relatively expensive encoding system, wouldn't it be better to keep corba > > out of kernel space and talk something which is a simple and cleaner encoding > p9fs exists. I didn't see these patches since August, but probably I can poke > Roman into porting it to the current tree. 9P is quite simple and unlike > CORBA it had been designed for taking kernel stuff to userland. Besides, > authors definitely understand UNIX... One thing that you might want to mention Alexander: 9P is not a general communications protocol. In fact, it doesn't work very well across the internet at all. To get decent performance, the Plan9 group (which, is a very cool group. :) has to specify a new protocol that competes with TCP on the level of complexity (IL: http://plan9.bell-labs.com/sys/doc/il/il.html) Also, 9P is a general communications framework only in the context of Plan9 itself. In reality it only applys directly/well to filesystem related issues... the reason it works well in Plan9 is that _everything_ is a file (part of the beauty of plan9). With some elbow grease, 9P could probably be made to work in the kORBit framework. It's not even that big of a deal: it just takes time. Believe me when I say that IIOP is not a very good user->kernel communications mechanism. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
/me trims down CC list... > Local? Funny. It lives atop of TCP or IL quite fine. What's > even funnier, I can use it to export /proc from CPU server to workstation > and use _that_ for remote debugging. Ditto for window system. Ditto for > DNS. Ditto for plumber. No, not on Linux... No not on linux. No definately not on Windows, no not on the BSD's either. Oops, wasn't interoperability an important part of the Linux kernel design? Didn't we want to use and follow and define _real_ standards? > What you do is mapping your RPC to hierarchical namespace. After > that... What extensibility do you need? You can use every utility written > for core UNIX API (open()/read()/write()/close()) as a client. No need > to reinvent the wheel... Heh, that's interesting. _getting_ the data isn't hard in 9P, actually using it or displaying it in a meaningful way commonly requires something more than cat. :) The overriding problem with a simplistic 9P port is that linux kernel functionality is not very well modularized. In fact, this prevents wonderful CORBA integration as well: An agressive filesystem (for example) often likes to touch the buffer cache or even the VM layer (*cough*) directly. Some don't use the prescribed interfaces, because nothing enforces it. 9P works wonderfully for Plan9 because they had the luxery of redefining/rewriting the whole OS, and the whole interaction with user space processes. We, unfortunately, don't have that possibility. :) Please don't get me wrong. I'm not opposed to other ideas, and 9P may in fact turn out to be a very nice protocol that would be able to support much kORBit level functionality). I do maintain that by writing a custom user->kernel marshalling library, you could obtain better peformance than 9P though, because you could take advantage of lots of machine specific optimizations. Hell you could even pass things in register if you'd have <= 4 args. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > Err shame on you, don't forget about lcall and exceptions, and interrupts, > > and... That is technically more than _o_n_e_ "entry point". :) Oh wait, > > what about sysenter/exit too? :) > OK, you got me on lcall (however, that's iBCS-only, IIRC), but the rest... > what the hell does userland to interrupts? OK, make it 2 - pagefault > can be arguably used in that way. The reason that I considered exceptions and interrupts is that often, exceptions get reflected as signals to the running processes (SIGSEGV, SIGFPE, SIGILL, others?), and interrupts can wake up processes (from sys_poll among others). I was considering more of the user->kernel and kernel->user transitions... anyways, that's really besides the point. :) > > this wonderful design we get all kinds of stuff like sys_oldumount vs > > sys_umount and others... > Check how often anything uses the majority of that stuff... Correct, it's for backwards compatibility with old programs (for example libc5 uses a lot of those "old" syscalls). > > > Yes, standard RPC mechanism would be nice. No, CORBA is not a good > candidate - > > too baroque and actually known to lead to extremely > tasteless APIs being > > implemented over it. Yes, I mean GNOME. So > Check 9P and compare. Really. Section 5 of Plan 9 manpages. Available on > plan-9.bell-labs.com/sys/man/ That's fine. Since the server is down (or the URL is bad), can you please give me an example of how 9P is better than CORBA? I freely admit to not knowing much about 9P... how much do you know about CORBA (aside from your opinion that GNOME uses it, and therefore it is bad. ;)? > > without breaking backwards compatibility). Please don't tell me that OOP > > is bad... or else we will have the eviscerate the VFS layer from the > > kernel (amount other subsystems)... :) > OOP is a nice tool. However, it's a tool that has incredible potential of > shooting one's foot. It's wonderful if you have sane set of methods. And > that's a _big_ if. "Easily extensible" is not an absolutely good thing - > C++ wankers all over the world are busily proving it every day. Heck, they > make a living out of that. IOW, the problem with interface changes is _not_ > in converting the old code. It's in choosing the right changes. And that > part of the game can't be simplified. Oif. That's like telling someone that C is evil because it has for loops, and for loops can be used to write nasty code. "just write in assembler" he says. :) I would claim that someone could write a bad program (or shoot themselves in the foot) with any turing complete language. C++ definately give you more rope to do that with, but used wisely, it can also be nice. The trick is to just not have to work with other peoples C++ code. :) Hey, did I mention that kORBit and all its extensions are written in C? :) > > Like I mentioned in a previous email, CORBA does not preclude 9P. What > > it does buy you though, is compatibility with LOTS of preexisting CORBA > > tools. How much development infrastructure is there for 9P? I thought > > so. :) > All UNIX userland on the client side. lib9p on the server side (23Kb of sparse > C). Examples of use in servers - see the aforementioned site. Err... yeah, so you're effectively mapping UNIX/POSIX across 9P. That's not very creative, and you could do the same thing with CORBA. I ask again, "How much development infrastructure is there for 9P?". If you say "just use unix", then what is the point of 9P at all? (on linux). Linux already has most of posix (and some would claim all of the "good stuff" in posix.). > > For one of our demos, we ran a file server on a remote linux box (that we > > just had a user account on), mounted it on a kORBit'ized box, and ran > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > syscalls. If nothing else, it's pretty nifty what you can do in little > > code... > Duh. And what's new about that? The "new" part is that our servers were < 100 lines of code each. Compare that to kNFS. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > either. Oops, wasn't interoperability an important part of the Linux > > kernel design? Didn't we want to use and follow and define _real_ > > standards? > Erm... 9P stub exists for Linux. It exists for FreeBSD. I suspect that > it exists for other *BSD too - never checked that. Okay, so there are _stubs_ for these platforms. How many languages are there bindings for? What about Win32 (gross I know, but at least for now, it is important)? What about OS/2 what about Amiga and palm? CORBA is very popular for certain things... even inferno isn't nearly as big... > > Heh, that's interesting. _getting_ the data isn't hard in 9P, actually > > using it or displaying it in a meaningful way commonly requires something > > more than cat. :) > Same for any other RPC mechanism. Of course. Which is why CORBA is about putting STRUCTURE in that stream of random bytes coming over the wire. Why should I have to rewrite my marshalling and demarshalling code every time I want to write a server. read and write are fine. But sometimes I want a structure. Sometimes, my structures aren't laid out like C struct's either. What then? What if I want to send an "object" to you? > > nothing enforces it. 9P works wonderfully for Plan9 because they had the > > luxery of redefining/rewriting the whole OS, and the whole interaction > > with user space processes. We, unfortunately, don't have that > > possibility. :) > Notice that they _don't_ export random internal APIs to userland. And neither do we. Beautiful isn't it? :) > > Please don't get me wrong. I'm not opposed to other ideas, and 9P may in > > fact turn out to be a very nice protocol that would be able to support > > much kORBit level functionality). I do maintain that by writing a custom > > user->kernel marshalling library, you could obtain better peformance than > > 9P though, because you could take advantage of lots of machine specific > > optimizations. Hell you could even pass things in register if you'd have > > <= 4 args. :) > > You mentioned OOP, didn't you? Encapsulation is a good thing and what you > are talking about is "layering violations made Real Easy(tm)". > I simply don't see why _that_ is a good goal. I completely fail to see how I'm breaking encapsulation or making layering violations easy. Why I'm talking about is the fact that CORBA doesn't care how you ship bytes around: it's a higher level protocol that cares about shipping "things" around. Bytes are old school, "stuff" is the wave of the future. ;) heh. -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > plan-9.bell-labs.com/sys/man/ > Arrgh. s/plan-9/plan9/. My apologies. Cool, thanks, will read. :) > IDGI. What 9P gives is an RPC mechanism that uses normal (as in "named streams > of characters") representation on the client side and very light-weight > library on the server side. It looks like you are trying to do a mechanism > that would export arbitrary _internal_ kernel APIs. I really don't see > what you would want it for. Which is exactly why it doesn't work well for many applications. The problem is this: how do you get from a byte stream to a structured data stream? There are many answers: 1. Keep your data structures so simple, that it's obvious. Not a good choice. :) 2. Defined interfaces to the bytestream for every interface that you define. 3. Define a standard for inflating/deflating "things" into bytestreams. Oh wait, that's what corba does. :) The point of the matter is that CORBA gives you a nice, clean, well supported method of not having to write marshalling/demarshalling code every time you want to use an interface. Not only that, it lets you define "fast paths" for special cases (ie, with user level corba, if you are communicating between two objects in the same processes, you can do direct calls and do no marshalling). These fast paths do not interfere with the generality of the system at all. The 9P way of doing things is not fundementally new, they just applied the idea that "everything is a file" more broadly. What annoys me is that it is not immediately obvious how to "demarshall" the data that comes out of /dev/mouse for example. Combine that with the problem that /dev/mouse might change format in the future (okay stupid example, but you get the idea) to use floating point coordinates, and things certainly get hairy. Why reinvent the wheel countless times? -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: CORBA vs 9P
> > Okay, so there are _stubs_ for these platforms. How many languages are > > there bindings for? > Grr... Let's define the terms, OK? What is available: kernel code that > represents the client side of RPC as a filesystem. Userland clients do > not know (or care) about the mechanisms involved. But they DO CARE about the format of the data. > And files with structure are things of dreadful past. BTDT. > You really need to... work with an OS that would have and enforce > "structured files" to appreciate the beauty of ASCII streams. Ahhh, so ASCII streams are a wonderful thing. Are you an XML fan? :) > However, that's a different story. What I _really_ don't understand > is the need to export anything structured from kernel to userland. Okay, how about a few examples. How about /proc/meminfo? How about the "stat" structure? How about /proc/stat? You seem to be indicating that ASCII files are fine for general exportation of kernel information. The /proc filesystem begs to differ. One specific example is the /proc/meminfo file. Why is it that one field is 0 now? Ouch we can't change the format of the file because we'll break some program. Crap, you want to add a field, well, tough luck. The struct stat example is one _trivial_ example of "the need to export anything structured from kernel to userland". > IOW, I would really like to see a description of use of your > mechanism. If it's something along the lines of "let's take a network > card driver, implement it in userland and preserve the current API" - > see the comment about layering violations. You've taken an internal > API and exposed it to userland in all gory details. See also your own > comment about internal APIs being not convenient for such operations. I'm not trying to dictate interfaces. I'm not trying to tell people what to use this stuff for. I'm arguing that it's useful and that you can do very interesting things with it. > If it's something else - I wonder what kind of objects you are talking > about and why opaque stuff (== file descriptors) would not be sufficient. Opaque stuff is fine. I have no problem with file descriptors. They effectively solve the exact same class of problems that CORBA does, except that they add significant _API BLOAT_ because every little "method" that implements them gets a syscall. Oh yeah, they you get ioctl too. Funness. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> OK, now I'm completely confused. > * which complex data structures do you want to export from the kernel > in non-opaque way? > * which of those structures are guaranteed to remain unchanged? > * if you have userland-to-userland RPC in mind - why put anything > marshalling-related into the kernel? Okay, I think I did my best to completely confuse you. :) #1: CORBA objects _are_ opaque. #2 is irrelevant due to #1. #3: userland->userland is not the interesting part. We want kernel->userland or user->kernel, as the common (ie fast) case, but we also want to do client->client or kernel->kernel without anything breaking. > > /dev/mouse for example. Combine that with the problem that /dev/mouse > > might change format in the future (okay stupid example, but you get the > > idea) to use floating point coordinates, and things certainly get hairy. > HUH? OK, suppose it had happened. Do you really expect that you will not > need to change your applications? I mean, if you expected a bunch of YES. You should not have to change your apps at all. > ints and received a bunch of doubles... You either need to decide on > rounding (and it's a non-obvious question) or you need to change quite a > bit of code in your program. It goes way past the demarshalling, no > matter whether you use CORBA, 9P or printf/scanf. NO. You want leagacy program to "just get" rounded ints, and new programs to get the "full precision" of the floating point #'s. > OK, suppose you have a CORBA-based system and mouse drivers' API had been > changed - they really want to return floating point coordinates. How will > CORBA help you? Aside of making your programs scream aloud, that is. Okay, maybe this isn't a terrible example afterall. :) Consider this theoretical API for a mouse, that I'm certainly not saying is wonderful or perfect, just consider it to be an example (on the fly), and no it doesn't let you poll for mouse activity, this is just off the cuff: Original interface for the mouse: interface mouse { void getPosition(out long X, out long Y); }; So if you get the mouse interface in CORBA (which is an opaque object), you get back something that you can make requests to, and, for example, get the coordinate the mouse is currently at. Okay, that's fine, 15 years later and after much legacy software has been developed, company X develops a new high precions floating point mouse. Well crap, don't want to change the interface. Now instead of getting that, you get the "mouse2" or "floaty mouse" interface (for sake of argument suppose that we actually did want to use FP arithmetic in the kernel, that's an artifact of a bad example, not a bad point :): interface floatymouse : extends mouse { void getFloatyPosition(out float X, out float Y); } Now people that get the "/dev/mouse" interface get a floatymouse. Does this break all that leagacy mouse wielding code? No, because we used inheritance, the floatymouse is a superset of the mouse, and the original interface still works. The server/kernel side of the floaty mouse just chops off the decimal places when you use the getPosition method. This is the kind of stuff that I'm talking about. The concept that a mouse is more than a byte stream... that there is actual structure that must be understood before a datastream can be used. Yes, I am very aware that there are other ways of doing this, but CORBA is a very general & powerful system that is also quite mature (unlike kORBit. ;) I hope this helps clarify what the heck I'm talking about. :) -Chris btw, just imagine how much cleaner the mount interface could be... ;) http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: CORBA vs 9P
I think that I addressed most if not all of this email in my previous one... let me know if I missed something. -Chris btw, thanks for putting up with me, I know I can be obstinate sometimes. :) On Wed, 13 Dec 2000, Alexander Viro wrote: > > > On Wed, 13 Dec 2000, Chris Lattner wrote: > > > > > Okay, so there are _stubs_ for these platforms. How many languages are > > > > there bindings for? > > > Grr... Let's define the terms, OK? What is available: kernel code that > > > represents the client side of RPC as a filesystem. Userland clients do > > > not know (or care) about the mechanisms involved. > > > > But they DO CARE about the format of the data. > > And how would CORBA help here? Because format changes are usually coming > from the _contents_ changes. And if you don't care about the contents - > why the hell do you retrieve the object int the first place? > > > > And files with structure are things of dreadful past. BTDT. > > > You really need to... work with an OS that would have and enforce > > > "structured files" to appreciate the beauty of ASCII streams. > > > > Ahhh, so ASCII streams are a wonderful thing. Are you an XML fan? :) > > No, thanks. > > > > However, that's a different story. What I _really_ don't understand > > > is the need to export anything structured from kernel to userland. > > > > Okay, how about a few examples. How about /proc/meminfo? How about the > > "stat" structure? How about /proc/stat? You seem to be indicating that > > ASCII files are fine for general exportation of kernel information. The > > Yes, _if_ you take care to think what you are exporting. /proc/meminfo is > a shi..ning example of _not_ doing that over many years. > > > /proc filesystem begs to differ. One specific example is the > > /proc/meminfo file. Why is it that one field is 0 now? Ouch we can't > > change the format of the file because we'll break some program. Crap, you > > want to add a field, well, tough luck. > > Oh, cool. So CORBA would magically change the definition of the structure in > your (C/Modula-3/APL/COBOL) programs. How? And what would happen with the > code that used to access the field in question? > > > The struct stat example is one _trivial_ example of "the need to export > > anything structured from kernel to userland". > > It's a trivial example of "why you need to think before deciding what to > export". > > > > IOW, I would really like to see a description of use of your > > > mechanism. If it's something along the lines of "let's take a network > > > card driver, implement it in userland and preserve the current API" - > > > see the comment about layering violations. You've taken an internal > ^ > > > API and exposed it to userland in all gory details. See also your own > > > comment about internal APIs being not convenient for such operations. > > > > I'm not trying to dictate interfaces. I'm not trying to tell people what > > to use this stuff for. I'm arguing that it's useful and that you can do > > very interesting things with it. > > And when interface changes, you do what, exactly? > > > > If it's something else - I wonder what kind of objects you are talking > > > about and why opaque stuff (== file descriptors) would not be sufficient. > > > > Opaque stuff is fine. I have no problem with file descriptors. They > > effectively solve the exact same class of problems that CORBA does, except > > that they add significant _API BLOAT_ because every little "method" that > > implements them gets a syscall. > > Huh? Could you elaborate, please? > > -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > NO. You want leagacy program to "just get" rounded ints, and new programs > > to get the "full precision" of the floating point #'s. > What rounded ints? Rounded to zero? To nearest integer? To plus or minus > infinity? Does program have something to say here? The exact same thing that older mouse drivers did. In this particular case, it doesn't matter anyways, because it's a simple example. If you REALLY care about rounding those coordinates a certain way, it's trivial to add an extra component to the interface. > ... > My mouse example > ... > Oh, great. So we don't have to care about formatting changes. We just > have to care about the data changes. IOW, we are shielded from the > results of changes that should never happen in the first place. And the > benefit being...? What the hell are you talking about? Did you even read my example? I was giving an example of extending an API, adding new functionality to it. > Notice that we could equally well add /dev/floatymouse and everything that > worked with old API would keep working. The only programs that would need > to know about floats would be ones that would... need to know about floats > since they want to work with them. Wonderful solution. This is exactly what you would _have_ to do. Are you aware of why devfs came into existence? It is because there are already so damn many device nodes in /dev. Lets keep adding more. Why is it (for example) that on my machine (mandrake 7 install, no devfs), I automatically get inodes for ISDN & PPP!?! Well, because ISDN has a slightly different interface, and you're talking to a different driver that is not transparently hidden to the user. Why can't I just open /dev/net0 and get the first network device? Because we have so many inconsistent, poorly design, inextensible interfaces laying around, thats why. > Notice also that I can say ls /dev/*mouse* and get some idea of the files > there. I can't do that for your interfaces. And you know what? I can't do ls /dev/*net* and get all the network devices either. Actually, one of the very cool things about CORBA is that you can BROWSE/LIST/SEARCH all objects currently instantiated. Being able to browse all in kernel objects would be very cool. > The point being: if your program spends efforts on marshalling it would > better _do_ something with the obtained data. And then we are back to > the square 1. Uh huh, go ahead and read the example I sent to you. > Returning to your example, I could not tee(1) the stream into file for > later analysis. Not unless I write a special-case program for intercepting > that stuff. I don't see why it is a good thing. On the contrary, it would be pretty easy to do something like that with CORBA. No you wouldn't be able to use tee, but why would you want to tee a binary data file? The only reason that tee works in this situation is because it's agnostic to the format of the data coming off the line... your analysis program would have to have special purpose code to parse the file. EVERY consumer of "mouse data" would have to parse the file. That seems pretty silly to me. > I also don't see where the need in new system calls (or ioctls, same shit) > comes from. Notice that your way is much closer to new system call than > read()/write() of the right stuff. A new syscall was one example. It would be very simple to implement this by making _yet_another_ device node in /dev and issue reads and writes to it. That's more of a syntactic issue than a symantic one. > As for the proc/meminfo... What would you do to a userland programmer who > had defined a structure like that? Let's see: way too large, ugly as hell, > many fields are almost guaranteed to become meaningless at later point... > It was not designed, it got accreted. And that's very mild description - > judging by results it might as well be s/ac/ex/. There is no provision in CORBA that prevents the creation of ugly interfaces. There is no provision in C that prevents them either, as witnessed by this example. At least with CORBA, I can extend the interface to actually return meaningful results while still maintaining backwards compatibility with programs that don't actually look at that field (as is done now by sticking the 0 in there). The trick is having enough foresight to design robust interfaces. I'm not saying that I'm an expert at that, and obviously nobody can be. What I _am_ saying is that CORBA/kORBit gives you flexibility in how you do interfaces, and it lets you "future proof" them (obviously only within reason). Adding a new device node everytime an API gets changed is like defining a different API for every hard disk that exists. Seems kinda silly. -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > 1. kORBit adds about 150k of code to the 2.4t10 kernel. > > 2. kNFS adds about 100k of code to the 2.4t10 kernel. > So can you implement a kNFS server in kORBit that takes > less than 50kB of RAM? Otherwise it's still a contributor > to bloat and this argument won't work ;) Actually the kORBitNFS server would have to take -50K of code to break even. :) The point was that kORBit lets you do a lot more... so hopefully that 50k of generality gives you something. :) > I guess it's time to stop the flaming and to see what can > be achieved using kORBit. The people who favour kORBit should > IMHO be left alone and given the opportunity to show what can > be achieved with kORBit ... if they don't achieve anything, > the nay-sayers can always claim their "victory"; if something > useful comes out the kORBit people can claim usefulness. Agreed! -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ORBit speed measure
>> There is a large perception of CORBA being slow, but for the most part >> it is unjustified. > Well, I've measured using function calls through ORBit is 300 times > slower than using dynamic loading. > ... > Which gives me the dl is about 333 times faster than ORBit. You leave so many details out, and you sortof miss the point with my statement. The idea is that CORBA doesn't specify a transport to use _at all_ except when doing inter-orb communication (which you basically have to use IIOP to be compatible). This is why, for example, if you ior-decode ORBit IORs that they contain two parts: an IIOP portion, and a Unix domain socket portion. Basically ORBit is optimizing the intramachine case, because it is very common for purposes that ORBit is mainly used: GNOME. You didn't specify, but I assume that you were using the Unix sockets transport for your benchmark. I am also assuming that the two things communicating were in different processes. If you wanted to make a more fair test case, you would have to use some form of IPC mechanism to communicate between two distinct processes. Comparing a function call to an RPC mechanism will never look good. :) There are many other optimizations that one can make the transport faster that ORBit doesn't implement. For example, you could mmap (shared) data buffers between the two processes communicating (of course, you still need to wake processes up, which is why it hasn't been done yet), or you could use pipes (if you can control forkness) or any number of things. If you are communicating with a server in the same process as your client (because it was dynamically loaded) you can even do a direct function call. When I said that "for the most part it is unjustified" to think that CORBA is slow, what I meant is that CORBA isn't slow: implementations are. Just look at the difference between ORBit and all those other orbs out there... orders of magnitude speed differences, with little or no functionality differences. What CORBA gives you is the power to have optimizations for important special cases (in kORBit's case, one of those would be User->kernel), WITHOUT sacrificing the big picture or the ability to be interoperable... I am not claiming that kORBit is where it needs to be right now. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> >that is not transparently hidden to the user. Why can't I just open > >/dev/net0 and get the first network device? Because we have so many > >inconsistent, poorly design, inextensible interfaces laying around, thats > >why. > This is a bad example, but a (perhaps?) good point. It seems it should be > possible to implement an unlimited number of TCP/IP devices, each Part of my argument is that although every kernel interface boils down to something as simple as ioctl (where you have lots of api's multiplexed onto one data sink, int 0x80 is another example, kORBit is another), some ways of doing this multiplexing are better than others. I look at it as a hierarchy: 1. CORBA/kORBit 2. int 80/lcall 3. ioctl Each level is more structured than those below it. Each interface is also "cleaner" than those below it. I don't think that anyone would argue that we should replace int 80 and friends with an interface in the spirit of ioctl (even though it would be functionally identical). > representing a connection, and each connected, disconnected, etc. by > ioctls... Of course, I could be way wrong about this ioctl's... they come up again. Everytime that I look at the beautiful, clean, abstract interface that Unix exposes... it makes me happy. But unfortunately, if you look closer, you realize that the Unix API isn't really clean or nice at all... there is this dumping ground for odds and ends that don't fit into the standard model. So when you show someone the standard model, it looks clean and pretty... but someone always forgets to mention ioctl. I would claim that ioctl is one of the biggest reasons that kORBit needs to exist. See below. > >I can't do ls /dev/*net* and get all the network > >devices either. Actually, one of the very cool things about CORBA is that > Different network devices are _very_ different... You can't broadcast on a > modem, for instance, nor can you use promiscuous mode on a modem... You > also can't do byte-by-byte raw mode on an ethernet... Precisely. But they do all have very common interfaces. For example, the standard network interface would probably have "transmit/receive block" and "getstatus" commands. This does not mean that they cannot implement other interfaces, however. Imagine this situation for writing your packet sniffer: 1. You open up /dev/net0 and get the Network interface 2. You query the network interface to see if it has an implementation of the promiscuous interface. 3. If not, you bail, because it doesn't support it. 4. If, you go ahead and use it. Contrast that to: 1. Survey all the different interfaces that are known (at development time of course) to have promiscuous interfaces. 2. Find out which ones are active. 3. Depending on what kind of interface it is, load a library that can understand the byte format comming off the device. 4. Use that library. The problems with the second (currently used) approach is that the APPLICATION writer has to keep up with new hardware developments and new interfaces. They have to understand and code parsers for the bytestreams coming from each device, which (as you mentioned) are all different, because they all support different (although overlapping) extensions. All of these calls that currently get dumped into ioctl (because they are not important enough to warrant an API function at the top level) suddenly become well structure and well designed interfaces. For example, if your device doesn't support the terminal handling ioctls the system has to basically figure that out and report errors on it. With CORBA/kORBit, you would just not implement that interface... so if someone asked for it, they would get a null pointer to the interface, which they check for and realize that it's impossible to do terminal stuff on. isatty (an example of an IOCTL wrapper) suddenly becomes implicit and the API is cleaner... I have no problem with the "unix way", but I do think that it can be augmented in some ways... -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > There is a large perception of CORBA being slow, but for the most part it > > is unjustified. I believe that the act of _designing_ a completely new > CORBA is slow compared to some of the other solutions. The question I was > trying to ask is whether you should put something smaller and faster into the > kernel space and leave CORBA in userland. What I'm trying to show is that CORBA itself is not slow. CORBA can be thought of as an IDEA, and the current implementations are suboptimal. For example, no CORBA implementation has been tweaked to run well in-kernel. Because of this, people will say that CORBA sucks and is slow, but that's not true. I will agree that IIOP (the standardized corba communication mechanism) is much slower and more complex than we would want for a generalized user->kernel communication mechanism. The nice thing about CORBA is that you can design your own transport to optimize things. Thus 99% of the time you could use a heavily optimized, very simple transport that doesn't do much. This is what you are asking for, and it make perfect sense. What CORBA buys you is the ability to use this streamlined interface without giving up full generality... > It's complex, it has security > implications surely it belongs talking something simple and fast to the kernel. First off, there are a lot of complex systems in the kernel. :) Second, there are not security implications above and beyond any normal kernel hacking. The reason that kORBit has security implications right now is that there _IS NO SECURITY_. Imagine implementing sys_read with no checks of what uid is running, and you get the idea. Actually security in CORBA can be done BETTER in kernel space than in user space, but I digress... > If you look at microkernels they talk a much simpler faster rpc protocol. Yes but for the most part they lose all of the advantages of CORBA as well. For example, every microkernel that I'm aware of can only talk to remote servers/clients that are running the same microkernel on the remote machine. There are no provisions for byte swapping if the endianness is incorrect or for having structured bytestreams (so that you can tell things about the raw bytes coming over the line without understanding them all). -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > Of course. Which is why CORBA is about putting STRUCTURE in that stream > > of random bytes coming over the wire. Why should I have to rewrite my > > marshalling and demarshalling code every time I want to write a > > server. read and write are fine. But sometimes I want a > > structure. Sometimes, my structures aren't laid out like C struct's > > either. What then? What if I want to send an "object" to you? > > Then I need to understand the object anyway. And Corba objects are horribly > over complex. Any lisp hacker will tell you there is only one type: a list. But alan, that's the beautiful thing. Given a CORBA object, you can understand its structure without knowing exactly what the contents are. You can effectively derive it's prototype just by inspecting it. The only difference between lisp and a CORBA object in this respect is that each item in the list is typed so that you have even more info about what to do with it. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > There is a large perception of CORBA being slow, but for the most part it > > > is unjustified. > > Really? I have that same perception but I can't claim that I've measured it. > On the other hand, I have measured the overhead of straight UDP, TCP, and > Sun RPC ping/pong tests and you can find the code for that in any version > of lmbench. It should be a 5 minute task for someone who groks corba to > do the same thing using the same framework. If someone wants to do it, > I'll guide them through the lmbench stuff. It's pretty trivial, start Urm... thanks for the offer... but you misunderstand me if you think that I'm claiming that kORBit is the ideal/fast implementation that everyone has been looking for. There is still much to be done. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > Oh, great. So we don't have to care about formatting changes. We just > > > have to care about the data changes. IOW, we are shielded from the > > > results of changes that should never happen in the first place. And the > > > benefit being...? > > > > What the hell are you talking about? Did you even read my example? I was > > giving an example of extending an API, adding new functionality to it. > > Yes, I did. What I don't understand is how kernel mechanism for marshalling > would make your life easier wrt changes. I gave a very simple example of how an interface could be designed and then later extended without breaking any user space programs. Please reread the example and tell me precisely what doesn't make sense, and I would be glad to explain it better. > > > Notice also that I can say ls /dev/*mouse* and get some idea of the files > > > there. I can't do that for your interfaces. > > And you know what? I can't do ls /dev/*net* and get all the network > > devices either. Actually, one of the very cool things about CORBA is that > > you can BROWSE/LIST/SEARCH all objects currently instantiated. Being > > able to browse all in kernel objects would be very cool. > So what's to stop you from letting readdir() do the work? WTF do we need > one more API for returning the list of objects? Urm.. that was the point. You don't add a new API (it's built into corba, along with introspection). Also, like I JUST MENTIONED, readdir will have no way of knowing what is a network device in the quagmire of stuff floating around in /dev. (ls *net* does actually call readdir ya know. Perhaps you should read up more on unix... [sorry couldn't help it. ;]) > > > The point being: if your program spends efforts on marshalling it would > > > better _do_ something with the obtained data. And then we are back to > > > the square 1. > > Uh huh, go ahead and read the example I sent to you. > Did that. So they you understand that no interface was broken even though more data is now available... > > > Returning to your example, I could not tee(1) the stream into file for > > > later analysis. Not unless I write a special-case program for intercepting > > > that stuff. I don't see why it is a good thing. > > On the contrary, it would be pretty easy to do something like that with > > CORBA. No you wouldn't be able to use tee, but why would you want to tee > > a binary data file? The only reason that tee works in this situation is > For the same reasons why I use tar, gzip, whatever. I don't _want_ to > invent a new utility every time when Joe Doe adds a new piece of interface. > Data is data is data. I can uuencode it and send to somebody who would > care to analize the bloody thing. Do you mean that I need to write a > special tool for that? For _every_ member of every interface somebody > decided to add? I don't think so. I really don't. That's fine, I don't blame you. Ya know what? I even agree with you (image that? hehe :). You seem to forget that a CORBA object is exactly the same thing: a stream of bytes. Actually it does better than that. It's a stream of bytes that you can introspect on and determine the structure of. Imaging this. You have a bytestream flowing out of some pipe/device/socket/file/whatever and you pipe it to a "Decode" program. The decode program spits this out: sequence < struct { float float }> This example is meant to illustrate that _in a general way_ you can decode the structure of the bytestream. (hence my claims about CORBA adding structure to a standard bytestream)... this is quite a bit more useful than trying to figure out what 'cat' is trying to tell you about binary data (especially if it's floating point numbers). > > line... your analysis program would have to have special purpose code to > > parse the file. EVERY consumer of "mouse data" would have to parse the > Or I would look at the size. Or I would say od and look at the result. Don't > tell me what to do with the data, when I'll need to parse it I will. And > if you expect me to bother with writing more stuff when generic tools would > work fine - too bad, I've been there, done that and I'm not coming back. Heh... I'm not trying to replace od or cat or tee or pipes. What I'm trying to do is give you a larger toolkit that gives you MORE power. Imagine you want to get the first float value (of each struct) of the above stream. Imagine a functional scripting language where you could do this (haskell syntax stolen gratitously): cat /dev/floatymouse | funcwith 'map fst' Which you could then point to the "decode" type program mentioned above, and you would now get: sequence > BTW, you cared about size of /dev, didn't you? /usr/bin choke-full of tee-foo, > tee-bar, yodda, yodda would be better? You wouldn't have to replace the standard utilities. Sorry to rain on your parade. > > file. That seems pretty silly to me. > Difference between good program and bad one: the
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > For one of our demos, we ran a file server on a remote linux box (that we > > just had a user account on), mounted it on a kORBit'ized box, and ran > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > syscalls. If nothing else, it's pretty nifty what you can do in little > > code... > > Cool! > > However, can you do one test for me? Do _heavy_ writes on kORBit-ized > box. That might show you some problems. Oh, and try to eat atomic > memory by ping -f kORBit-ized box. I'll give that a try when I get a chance. :) > I've always wanted to do this: redirect /dev/dsp from one machine to > another. (Like, I have development machine and old 386. I want all > programs on devel machine use soundcard from 386. Can you do that?) Yes. Definately. There are probably other ways of doing that... but one of the things we implemented was a "generic" character device... and we tested it by having a chardev server that basically reads from a "local" (to the server) character device, and forward it over CORBA. So this is already implemented! :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > cat /mnt/www/www.kernel.org/index.html > > can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came > to conclusion that web filesystem is not possible... (If you can't do Yes, if the server supports webDAV or something similar. > listings, it is not really filesystem; you could do > > cat /mnt/www/www.kernel.org_index.html as well, and that's easy to > do.) > > > and the CorbaFS userspace server takes care of loading the webpage and > > returning it to the kernel client. And these new filesystems don't > > take up any extra space in the kernel, since they all talk to the same > > CorbaFS kernel module! Not to mention being able to implement the > > filesystem in any language you like, debug the implementation in > > userspace, etc. > > codafs can do pretty much the same. Yes, but codaFS is specific to filesystems. kORBit, of course, can do much much more, in a very uniform way. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC] bloody mess with __attribute__() syntax
On Thu, 5 Jul 2007, Al Viro wrote: On Thu, Jul 05, 2007 at 09:41:55AM -0700, Linus Torvalds wrote: Note that gcc rules for __attribute__() (and that's the only source of rules we _have_ for the damn thing) clearly say that int __user *p; is the same thing as int *__user p; Quick question: is there some reason why we have to honor the crazy gcc rules, and cannot try to convince gcc people that they are insane? AFAICS, they started with storage-class-like attributes. Consider e.g. always_inline or section; these are not qualifiers at all and you want to have static __attribute__((always_inline)) int foo(int *p); interpreted with attribute applied to foo, not to its return type. This is true, but I don't think this is related. attributes in GCC can apply either to types or two decls. In this case, the always_inline attribute is being applied to the decl, but other attributes could be applied to the return type. -Chris -- http://nondot.org/sabre/ http://llvm.org/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/