ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-08 Thread Chris Lattner


This email is here to announce the availability of a port of ORBit (the
GNOME ORB) to the Linux kernel.  This ORB, named kORBit, is available from
our sourceforge web site (http://korbit.sourceforge.net/).  A kernel ORB
allows you to write kernel extensions in CORBA and have the kernel call
into them, or to call into the kernel through CORBA.  This opens the door
to a wide range of experiments/hacks:

* We can now write device drivers in perl, and let them run on the iMAC
  across the hall from you. :)
* Through the use of a LD_PRELOAD'd syscall wrapper library, you can
  forward system calls through CORBA to an arbitrary local/remote machine.
* CORBA servers are implemented as Linux kernel modules, so they may be
  dynamically loaded or unloaded from a running system at any time.  CORBA
  servers expose their IOR's through a /proc/corba filesystem.
* Filesystems may be implemented as remote CORBA objects and mounted on
  the local machine, by using 'mount -t corbafs -o IOR:... none /mnt/corba'

This are just some of the features available _RIGHT_NOW_ that are
supported by kORBit.  I'm sure that YOU can think of many more.

Implementation:
We implemented this port by providing a user-kernel mapping layer that
consists of providing standard system header files for the "user" code to
#include.  In these header files, we do the mapping required.  For
example, we implement a stdio.h that #defines printf to printk (as a
trivial example).  Only user level code sees or uses these wrappers... all
of our modifications to the Linux kernel are contained within the
linux/net/korbit subdirectory.  

This is currently implemented with a 2.4.0test10 kernel, although forward
porting should be very easy.  This project was implemented as a cs423
semester project by Chris Lattner, Fredrik Vraalsen, Andy Reitz, and Keith
Wessel at the University of Illinois @ Urbana Champaign.

Unresolved issues:
* Our poll model is not optimial.  Currently we actually do a real poll on
  a (struct socket *) set.  This causes relatively high latencies (on the
  order 1 second, worst case) for CORBA requests.  Our waitqueues are not
  working quite as well as they should.  :)
* Security is completely unimplemented.  Someone could use corba
  interfaces to read any file on your system, for example (if the
  CORBA-FileServer module is installed).  Thus, this is really more for
  prototyping and development than actual real world use. :)

If you have any questions or comments, please feel free to contact us at:

Chris Lattner, Fredrik Vraalsen, Andy Reitz, Keith Wessel
[EMAIL PROTECTED]

btw, yes we are quite crazy, but what good is it to be normal and
conformist afterall?  :)




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-12 Thread Chris Lattner


On Sat, 9 Dec 2000, Mohammad A. Haque wrote:

 It was just an example. Basically, you'd be able to do in with just
 about any language that has ORBit bindings.
 
 Ben Ford wrote:
  Why would you *ever* want to write a device driver in perl???
 

Precisely... but also, there could be a case where perl would make
sense.  Consider an FTP filesystem.  There performance is not dictated by
the speed of the language, it's limited by bandwidth.  It could make sense
to write your almighty FTPfs like this:

1. Prototype it in Perl, get all the bugs out.
2. Rewrite in C in userspace, get all the bugs out.
3. recompile/relink in kernel space with no source modifications
4. ship product.  :)

The great thing that kORBit buys you is insulation of kernel space from
the drivers that are running... kinda like a microkernel.  I'm not going
to start a flamewar here about Linux and microkernels, but when doing
initial development work for a driver, the test/crash/reboot/fsck cycle is
a real pain (okay, maybe journalling helps a little, but you get the
idea).  What we're offering goes more like this:

1. Boot kernel
2. Install corbafs module for example
3. Start test filesystem in user space
4. mount test user space filesystem
5. test it, oh crap, it segfaulted.
6. CorbaFS gets exceptions trying to communicate to server, which it
relays to the kernel as -errno conditions.
7. You safely unmount corbafs
8. fix your bug
9. goto step #2.

Which is arguably nicer.  :)

The whole idea is that a bastard driver shouldn't take your kernel down if
you know it to be unreliable... if you trust the driver, then by all
means, don't use kORBit.  Also, kORBit is useful when you don't WANT
something in the kernel... and I won't bring up the whole user level
filesystem debate again... :)

-Chris

http://www.nondot.org/~sabre/os/
http://korbit.sourceforge.net/

 -- 
 
 =
 Mohammad A. Haque  http://www.haque.net/ 
[EMAIL PROTECTED]
 
   "Alcohol and calculus don't mix. Project Lead
Don't drink and derive." --Unknown  http://wm.themes.org/
[EMAIL PROTECTED]
 =
 


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


  Don't worry about kORBit.  Like most open source projects, it will simply
  die out after a while, because people don't find it interesting and there
  is really no place for it.  If it becomes useful, mature, and refined,
  however, it could be a very powerful tool for a large class of problems
  (like moving code OUT of the kernel).
 
 I do have one sensible question. Given that corba is while flexible a 
 relatively expensive encoding system, wouldn't it be better to keep corba
 out of kernel space and talk something which is a simple and cleaner encoding

Very good point.  I think this distills down to common misconception of
CORBA: that it must be slow.  Actually, a LOT of research has been done
to improve CORBA performance, but you have to keep in mind one very
IMPORTANT THING:

CORBA does NOT dictate a transport or data format for the data going out
"over the wire".

CORBA does specify one "standard" used for interoperability: GIOP.  It
also specifies one important incarnation of GIOP: IIOP (GIOP over
IP).  kORBit currently uses this to communicate from user space to kernel
space... obviously this is not ideal.  :)

There is absolutely NOTHING that prevents us from defining our own "data
marshalling" protocol/algorithm that is lighter weight than IIOP.  In
fact, we had planned to implement that for our project, but of course, ran
out of time.  It would be relatively straight forward to define this
extension in the ORBit code base, and in fact "ORBit 2" is striving to
make this much much easier than it is now (basically sorting out IIOP
specific code from the main code base better, and generally cleaning
things up).  Currently, ORBit supports IIOP and its own proprietary (as
far as I know) Unix domain socket protocol (obviously only works
intramachine).

There is a large perception of CORBA being slow, but for the most part it
is unjustified.  I believe that the act of _designing_ a completely new
protocol, standardizing it, and making it actually work would be a huge
process that would basically reinvent CORBA (obviously some of the design
decisions could be made differently, but all the same issues would have be
dealt with).

CORBA, today, gives us superior interoperability (through IIOP), with
extensibility for the future.  As Alexander Viro mentions, 9P may be a
better protocol for local communications... so CORBA gives us a framework
to try that out with.  Even IIOP is smart enough to, for example, only do
endian swapping of data if the two machines actually differ... so at least
it doesn't specify a "network" byte order. 

CORBA isn't ideal for all applications, it certainly isn't the ideal IPC
or RPC mechanism (the L4 designers would have a laughing fit if you tried
to use it for _general_ IPC), but it is very useful for a lot of
things.  kORBit is very pre-alpha code, it works, but has lots of room for
enhancements, experiementation, and toying around with.  

That's what makes it cool.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


  Err... how about this:  Give me two or three kORBit syscalls and I can get
  rid of all the other 100+ syscalls!  :) 

 Like it ioctl() does it? Number of entry points is _not_ an issue. Diversity
 of the API is. Technically, kernel has 1 (_o_n_e_) entry point as far as
 userland is concerned. int 0x80 on x86. Can't beat that, can you?

Err shame on you, don't forget about lcall and exceptions, and interrupts,
and... That is technically more than _o_n_e_ "entry point".  :)  Oh wait,
what about sysenter/exit too? :)

No I can't beat that.  But if you look at the hack job of a system call
table we have, you can see that there is no _really_ standard way of
passing parameters.  Oh sure, most of the time, stuff is passed in
registers.  Sometimes we get a pointer to an argument struct.  Because of
this wonderful design we get all kinds of stuff like sys_oldumount vs
sys_umount and others...

 Yes, standard RPC mechanism would be nice. No, CORBA is not a good candidate -
 too baroque and actually known to lead to extremely tasteless APIs being
 implemented over it. Yes, I mean GNOME. So sue me.

Hrm... because I'm stupid, please explain how CORBA is too baroque...  I
have no problem with you not liking GNOME... you're a kernel hacker, so
you're not supposed to like GUI's.  :)  [just kidding!!!] CORBA doesn't
preclude nasty APIs any more than C does.  It also doesn't preclude *nice*
APIs that are upgradable and extensible in the future (and that means
without breaking backwards compatibility).  Please don't tell me that OOP
is bad... or else we will have the eviscerate the VFS layer from the
kernel (amount other subsystems)... :)

 I would take 9P over that any day, thank you very much.

Like I mentioned in a previous email, CORBA does not preclude 9P.  What
it does buy you though, is compatibility with LOTS of preexisting CORBA
tools.  How much development infrastructure is there for 9P?  I thought
so.  :)

For one of our demos, we ran a file server on a remote linux box (that we 
just had a user account on), mounted it on a kORBit'ized box, and ran
programs on SPARC Solaris that accessed the kORBit'ized linux box's file
syscalls.  If nothing else, it's pretty nifty what you can do in little
code...

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


  I do have one sensible question. Given that corba is while flexible a 
  relatively expensive encoding system, wouldn't it be better to keep corba
  out of kernel space and talk something which is a simple and cleaner encoding

 p9fs exists.  I didn't see these patches since August, but probably I can poke
 Roman into porting it to the current tree.  9P is quite simple and unlike
 CORBA it had been designed for taking kernel stuff to userland.  Besides,
 authors definitely understand UNIX...

One thing that you might want to mention Alexander: 9P is not a general
communications protocol.  In fact, it doesn't work very well across the
internet at all.  To get decent performance, the Plan9 group (which, is a
very cool group.  :) has to specify a new protocol that competes with TCP
on the level of complexity (IL: http://plan9.bell-labs.com/sys/doc/il/il.html)

Also, 9P is a general communications framework only in the context of
Plan9 itself.  In reality it only applys directly/well to filesystem
related issues... the reason it works well in Plan9 is that _everything_
is a file (part of the beauty of plan9).  

With some elbow grease, 9P could probably be made to work in the kORBit
framework.  It's not even that big of a deal: it just takes time.  Believe
me when I say that IIOP is not a very good user-kernel communications
mechanism.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


/me trims down CC list...

   Local? Funny. It lives atop of TCP or IL quite fine. What's
 even funnier, I can use it to export /proc from CPU server to workstation
 and use _that_ for remote debugging. Ditto for window system. Ditto for
 DNS. Ditto for plumber. No, not on Linux...

No not on linux.  No definately not on Windows, no not on the BSD's
either.  Oops, wasn't interoperability an important part of the Linux
kernel design?  Didn't we want to use and follow and define _real_
standards? 

   What you do is mapping your RPC to hierarchical namespace. After
 that... What extensibility do you need? You can use every utility written
 for core UNIX API (open()/read()/write()/close()) as a client. No need
 to reinvent the wheel...

Heh, that's interesting.  _getting_ the data isn't hard in 9P, actually
using it or displaying it in a meaningful way commonly requires something
more than cat.  :)

The overriding problem with a simplistic 9P port is that linux kernel
functionality is not very well modularized.  In fact, this prevents
wonderful CORBA integration as well: An agressive filesystem (for
example) often likes to touch the buffer cache or even the VM layer
(*cough*) directly.  Some don't use the prescribed interfaces, because
nothing enforces it.  9P works wonderfully for Plan9 because they had the
luxery of redefining/rewriting the whole OS, and the whole interaction
with user space processes.  We, unfortunately, don't have that
possibility.  :)

Please don't get me wrong.  I'm not opposed to other ideas, and 9P may in
fact turn out to be a very nice protocol that would be able to support 
much kORBit level functionality).  I do maintain that by writing a custom
user-kernel marshalling library, you could obtain better peformance than
9P though, because you could take advantage of lots of machine specific
optimizations.  Hell you could even pass things in register if you'd have
= 4 args.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


  either.  Oops, wasn't interoperability an important part of the Linux
  kernel design?  Didn't we want to use and follow and define _real_
  standards? 
 Erm... 9P stub exists for Linux. It exists for FreeBSD. I suspect that
 it exists for other *BSD too - never checked that.

Okay, so there are _stubs_ for these platforms.  How many languages are
there bindings for?  What about Win32 (gross I know, but at least for now,
it is important)?  What about OS/2 what about Amiga and palm?  CORBA is
very popular for certain things... even inferno isn't nearly as big...

  Heh, that's interesting.  _getting_ the data isn't hard in 9P, actually
  using it or displaying it in a meaningful way commonly requires something
  more than cat.  :)
 Same for any other RPC mechanism.

Of course.  Which is why CORBA is about putting STRUCTURE in that stream
of random bytes coming over the wire.  Why should I have to rewrite my
marshalling and demarshalling code every time I want to write a
server.  read and write are fine.  But sometimes I want a
structure.  Sometimes, my structures aren't laid out like C struct's
either.  What then?  What if I want to send an "object" to you?

  nothing enforces it.  9P works wonderfully for Plan9 because they had the
  luxery of redefining/rewriting the whole OS, and the whole interaction
  with user space processes.  We, unfortunately, don't have that
  possibility.  :)
 Notice that they _don't_ export random internal APIs to userland.

And neither do we.  Beautiful isn't it?  :)

  Please don't get me wrong.  I'm not opposed to other ideas, and 9P may in
  fact turn out to be a very nice protocol that would be able to support 
  much kORBit level functionality).  I do maintain that by writing a custom
  user-kernel marshalling library, you could obtain better peformance than
  9P though, because you could take advantage of lots of machine specific
  optimizations.  Hell you could even pass things in register if you'd have
  = 4 args.  :)
 
 You mentioned OOP, didn't you? Encapsulation is a good thing and what you
 are talking about is "layering violations made Real Easy(tm)".
 I simply don't see why _that_ is a good goal.

I completely fail to see how I'm breaking encapsulation or making layering
violations easy.  Why I'm talking about is the fact that CORBA doesn't
care how you ship bytes around: it's a higher level protocol that cares
about shipping "things" around.  Bytes are old school, "stuff" is the wave
of the future.  ;)  heh.

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner

   plan-9.bell-labs.com/sys/man/
 Arrgh. s/plan-9/plan9/. My apologies.

Cool, thanks, will read.  :)

 IDGI. What 9P gives is an RPC mechanism that uses normal (as in "named streams
 of characters") representation on the client side and very light-weight
 library on the server side. It looks like you are trying to do a mechanism
 that would export arbitrary _internal_ kernel APIs. I really don't see
 what you would want it for.

Which is exactly why it doesn't work well for many applications.  The
problem is this: how do you get from a byte stream to a structured data
stream?  There are many answers:

1. Keep your data structures so simple, that it's obvious.  Not a good
choice.  :)
2. Defined interfaces to the bytestream for every interface that you
define.
3. Define a standard for inflating/deflating "things" into
bytestreams.  Oh wait, that's what corba does.  :)

The point of the matter is that CORBA gives you a nice, clean, well
supported method of not having to write marshalling/demarshalling code
every time you want to use an interface.  Not only that, it lets you
define "fast paths" for special cases (ie, with user level corba, if you
are communicating between two objects in the same processes, you can do
direct calls and do no marshalling).  These fast paths do not interfere
with the generality of the system at all.

The 9P way of doing things is not fundementally new, they just applied the
idea that "everything is a file" more broadly.  What annoys me is that it
is not immediately obvious how to "demarshall" the data that comes out of
/dev/mouse for example.  Combine that with the problem that /dev/mouse
might change format in the future (okay stupid example, but you get the
idea) to use floating point coordinates, and things certainly get hairy.

Why reinvent the wheel countless times?

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: CORBA vs 9P

2000-12-13 Thread Chris Lattner


  Okay, so there are _stubs_ for these platforms.  How many languages are
  there bindings for?
 Grr... Let's define the terms, OK? What is available: kernel code that
 represents the client side of RPC as a filesystem. Userland clients do
 not know (or care) about the mechanisms involved.

But they DO CARE about the format of the data.

   And files with structure are things of dreadful past. BTDT.
 You really need to... work with an OS that would have and enforce
 "structured files" spit to appreciate the beauty of ASCII streams.

Ahhh, so ASCII streams are a wonderful thing.  Are you an XML fan?  :)

   However, that's a different story. What I _really_ don't understand
 is the need to export anything structured from kernel to userland.

Okay, how about a few examples.  How about /proc/meminfo?  How about the
"stat" structure?  How about /proc/stat?  You seem to be indicating that
ASCII files are fine for general exportation of kernel information.  The
/proc filesystem begs to differ.  One specific example is the
/proc/meminfo file.  Why is it that one field is 0 now?  Ouch we can't
change the format of the file because we'll break some program.  Crap, you
want to add a field, well, tough luck.

The struct stat example is one _trivial_ example of "the need to export
anything structured from kernel to userland".

   IOW, I would really like to see a description of use of your
 mechanism. If it's something along the lines of "let's take a network
 card driver, implement it in userland and preserve the current API" -
 see the comment about layering violations. You've taken an internal
 API and exposed it to userland in all gory details. See also your own
 comment about internal APIs being not convenient for such operations.

I'm not trying to dictate interfaces.  I'm not trying to tell people what
to use this stuff for.  I'm arguing that it's useful and that you can do
very interesting things with it.

 If it's something else - I wonder what kind of objects you are talking
 about and why opaque stuff (== file descriptors) would not be sufficient.

Opaque stuff is fine.  I have no problem with file descriptors.  They
effectively solve the exact same class of problems that CORBA does, except
that they add significant _API BLOAT_ because every little "method" that
implements them gets a syscall.  Oh yeah, they you get ioctl
too.  Funness.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


 OK, now I'm completely confused. 
   * which complex data structures do you want to export from the kernel
 in non-opaque way? 
   * which of those structures are guaranteed to remain unchanged?
   * if you have userland-to-userland RPC in mind - why put anything
 marshalling-related into the kernel?

Okay, I think I did my best to completely confuse you.  :)  #1: CORBA
objects _are_ opaque.  #2 is irrelevant due to #1.   #3: userland-userland 
is not the interesting part.  We want kernel-userland or user-kernel, as
the common (ie fast) case, but we also want to do client-client or
kernel-kernel without anything breaking.

  /dev/mouse for example.  Combine that with the problem that /dev/mouse
  might change format in the future (okay stupid example, but you get the
  idea) to use floating point coordinates, and things certainly get hairy.

 HUH? OK, suppose it had happened. Do you really expect that you will not
 need to change your applications? I mean, if you expected a bunch of

YES.  You should not have to change your apps at all.

 ints and received a bunch of doubles... You either need to decide on
 rounding (and it's a non-obvious question) or you need to change quite a
 bit of code in your program. It goes way past the demarshalling, no
 matter whether you use CORBA, 9P or printf/scanf.

NO.  You want leagacy program to "just get" rounded ints, and new programs
to get the "full precision" of the floating point #'s.

 OK, suppose you have a CORBA-based system and mouse drivers' API had been
 changed - they really want to return floating point coordinates. How will
 CORBA help you? Aside of making your programs scream aloud, that is.

Okay, maybe this isn't a terrible example afterall.  :)  Consider this
theoretical API for a mouse, that I'm certainly not saying is wonderful or
perfect, just consider it to be an example (on the fly), and no it
doesn't let you poll for mouse activity, this is just off the cuff:

Original interface for the mouse:

interface mouse {
 void getPosition(out long X, out long Y);
};

So if you get the mouse interface in CORBA (which is an opaque object),
you get back something that you can make requests to, and, for example,
get the coordinate the mouse is currently at.  Okay, that's fine, 15 years
later and after much legacy software has been developed, company X
develops a new high precions floating point mouse.  Well crap, don't want
to change the interface.  Now instead of getting that, you get the
"mouse2" or "floaty mouse" interface (for sake of argument suppose that
we actually did want to use FP arithmetic in the kernel, that's an
artifact of a bad example, not a bad point :):

interface floatymouse : extends mouse {
  void getFloatyPosition(out float X, out float Y);
}

Now people that get the "/dev/mouse" interface get a floatymouse.  Does
this break all that leagacy mouse wielding code?  No, because we used
inheritance, the floatymouse is a superset of the mouse, and the original
interface still works.  The server/kernel side of the floaty mouse just
chops off the decimal places when you use the getPosition method.  

This is the kind of stuff that I'm talking about.  The concept that a
mouse is more than a byte stream... that there is actual structure that
must be understood before a datastream can be used.

Yes, I am very aware that there are other ways of doing this, but CORBA is
a very general  powerful system that is also quite mature (unlike
kORBit.  ;)

I hope this helps clarify what the heck I'm talking about.  :)

-Chris

btw, just imagine how much cleaner the mount interface could be... ;)

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


  NO.  You want leagacy program to "just get" rounded ints, and new programs
  to get the "full precision" of the floating point #'s.

 What rounded ints? Rounded to zero? To nearest integer? To plus or minus
 infinity? Does program have something to say here?

The exact same thing that older mouse drivers did.  In this particular
case, it doesn't matter anyways, because it's a simple example.  If you
REALLY care about rounding those coordinates a certain way, it's trivial
to add an extra component to the interface.

 ...
 My mouse example
 ...

 Oh, great. So we don't have to care about formatting changes. We just
 have to care about the data changes. IOW, we are shielded from the
 results of changes that should never happen in the first place. And the
 benefit being...?

What the hell are you talking about?  Did you even read my example? I was
giving an example of extending an API, adding new functionality to it.

 Notice that we could equally well add /dev/floatymouse and everything that
 worked with old API would keep working. The only programs that would need
 to know about floats would be ones that would... need to know about floats
 since they want to work with them.

Wonderful solution.  This is exactly what you would _have_ to do.  Are you
aware of why devfs came into existence?  It is because there are already
so damn many device nodes in /dev.  Lets keep adding more.  

Why is it (for example) that on my machine (mandrake 7 install, no devfs),
I automatically get inodes for ISDN  PPP!?!  Well, because ISDN has a
slightly different interface, and you're talking to a different driver
that is not transparently hidden to the user.  Why can't I just open
/dev/net0 and get the first network device?  Because we have so many
inconsistent, poorly design, inextensible interfaces laying around, thats
why.

 Notice also that I can say ls /dev/*mouse* and get some idea of the files
 there. I can't do that for your interfaces.

And you know what?  I can't do ls /dev/*net* and get all the network
devices either.  Actually, one of the very cool things about CORBA is that
you can BROWSE/LIST/SEARCH all objects currently instantiated.  Being
able to browse all in kernel objects would be very cool.

 The point being: if your program spends efforts on marshalling it would
 better _do_ something with the obtained data. And then we are back to
 the square 1.

Uh huh, go ahead and read the example I sent to you.

 Returning to your example, I could not tee(1) the stream into file for
 later analysis. Not unless I write a special-case program for intercepting
 that stuff. I don't see why it is a good thing.

On the contrary, it would be pretty easy to do something like that with
CORBA.  No you wouldn't be able to use tee, but why would you want to tee
a binary data file?  The only reason that tee works in this situation is
because it's agnostic to the format of the data coming off the
line... your analysis program would have to have special purpose code to
parse the file.  EVERY consumer of "mouse data" would have to parse the
file.  That seems pretty silly to me.

 I also don't see where the need in new system calls (or ioctls, same shit)
 comes from. Notice that your way is much closer to new system call than
 read()/write() of the right stuff.

A new syscall was one example.  It would be very simple to implement this
by making _yet_another_ device node in /dev and issue reads and writes to
it.  That's more of a syntactic issue than a symantic one.

 As for the proc/meminfo... What would you do to a userland programmer who
 had defined a structure like that? Let's see: way too large, ugly as hell,
 many fields are almost guaranteed to become meaningless at later point...
 It was not designed, it got accreted. And that's very mild description -
 judging by results it might as well be s/ac/ex/.

There is no provision in CORBA that prevents the creation of ugly
interfaces.  There is no provision in C that prevents them either, as
witnessed by this example.  At least with CORBA, I can extend the
interface to actually return meaningful results while still maintaining
backwards compatibility with programs that don't actually look at that
field (as is done now by sticking the 0 in there).

The trick is having enough foresight to design robust interfaces.  I'm not
saying that I'm an expert at that, and obviously nobody can be.  What I
_am_ saying is that CORBA/kORBit gives you flexibility in how you do
interfaces, and it lets you "future proof" them (obviously only within
reason).  Adding a new device node everytime an API gets changed is like
defining a different API for every hard disk that exists.  Seems kinda
silly.

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner

  1. kORBit adds about 150k of code to the 2.4t10 kernel.
  2. kNFS adds about 100k of code to the 2.4t10 kernel.

 So can you implement a kNFS server in kORBit that takes
 less than 50kB of RAM?  Otherwise it's still a contributor
 to bloat and this argument won't work ;)

Actually the kORBitNFS server would have to take -50K of code to break
even.  :)  The point was that kORBit lets you do a lot more... so
hopefully that 50k of generality gives you something.  :)

 I guess it's time to stop the flaming and to see what can
 be achieved using kORBit. The people who favour kORBit should
 IMHO be left alone and given the opportunity to show what can
 be achieved with kORBit ... if they don't achieve anything,
 the nay-sayers can always claim their "victory"; if something
 useful comes out the kORBit people can claim usefulness.

Agreed!

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ORBit speed measure

2000-12-14 Thread Chris Lattner


 There is a large perception of CORBA being slow, but for the most part
 it is unjustified.  

 Well, I've measured using function calls through ORBit is 300 times
 slower than using dynamic loading.
 ...
 Which gives me the dl is about 333 times faster than ORBit.

You leave so many details out, and you sortof miss the point with my
statement.  The idea is that CORBA doesn't specify a transport to use _at
all_ except when doing inter-orb communication (which you basically have
to use IIOP to be compatible).  This is why, for example, if you
ior-decode ORBit IORs that they contain two parts: an IIOP portion, and a
Unix domain socket portion.  Basically ORBit is optimizing the
intramachine case, because it is very common for purposes that ORBit is
mainly used: GNOME.

You didn't specify, but I assume that you were using the Unix sockets
transport for your benchmark.  I am also assuming that the two things
communicating were in different processes.  If you wanted to make a more
fair test case, you would have to use some form of IPC mechanism to
communicate between two distinct processes.  Comparing a function call to
an RPC mechanism will never look good.  :)

There are many other optimizations that one can make the transport faster
that ORBit doesn't implement.  For example, you could mmap (shared) data
buffers between the two processes communicating (of course, you still need
to wake processes up, which is why it hasn't been done yet), or you could
use pipes (if you can control forkness) or any number of things.  If you
are communicating with a server in the same process as your client
(because it was dynamically loaded) you can even do a direct function
call.

When I said that "for the most part it is unjustified" to think that CORBA
is slow, what I meant is that CORBA isn't slow: implementations are.  Just
look at the difference between ORBit and all those other orbs out
there... orders of magnitude speed differences, with little or no
functionality differences.

What CORBA gives you is the power to have optimizations for important
special cases (in kORBit's case, one of those would be User-kernel),
WITHOUT sacrificing the big picture or the ability to be interoperable...

I am not claiming that kORBit is where it needs to be right now.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


 that is not transparently hidden to the user.  Why can't I just open
 /dev/net0 and get the first network device?  Because we have so many
 inconsistent, poorly design, inextensible interfaces laying around, thats
 why.

 This is a bad example, but a (perhaps?) good point.  It seems it should be 
 possible to implement an unlimited number of TCP/IP devices, each 

Part of my argument is that although every kernel interface boils down to
something as simple as ioctl (where you have lots of api's multiplexed
onto one data sink, int 0x80 is another example, kORBit is another), some
ways of doing this multiplexing are better than others.  I look at it as a
hierarchy:

1. CORBA/kORBit
2. int 80/lcall
3. ioctl

Each level is more structured than those below it.  Each interface is also
"cleaner" than those below it.  I don't think that anyone would argue that
we should replace int 80 and friends with an interface in the spirit of
ioctl (even though it would be functionally identical).

 representing a connection, and each connected, disconnected, etc. by 
 ioctls...  Of course, I could be way wrong about this

ioctl's... they come up again.  Everytime that I look at the beautiful,
clean, abstract interface that Unix exposes... it makes me happy.  But
unfortunately, if you look closer, you realize that the Unix API isn't
really clean or nice at all... there is this dumping ground for odds and
ends that don't fit into the standard model.  So when you show someone the
standard model, it looks clean and pretty... but someone always forgets to
mention ioctl.

I would claim that ioctl is one of the biggest reasons that kORBit needs
to exist.  See below.

 I can't do ls /dev/*net* and get all the network
 devices either.  Actually, one of the very cool things about CORBA is that

 Different network devices are _very_ different...  You can't broadcast on a 
 modem, for instance, nor can you use promiscuous mode on a modem... You 
 also can't do byte-by-byte raw mode on an ethernet...

Precisely.  But they do all have very common interfaces.  For example, the
standard network interface would probably have "transmit/receive
block" and "getstatus" commands.  This does not mean that they cannot
implement other interfaces, however.

Imagine this situation for writing your packet sniffer:

1. You open up /dev/net0 and get the Network interface
2. You query the network interface to see if it has an implementation
   of the promiscuous interface.
3. If not, you bail, because it doesn't support it.
4. If, you go ahead and use it.

Contrast that to:

1. Survey all the different interfaces that are known (at development time
   of course) to have promiscuous interfaces.
2. Find out which ones are active.
3. Depending on what kind of interface it is, load a library that can
   understand the byte format comming off the device.
4. Use that library.

The problems with the second (currently used) approach is that the
APPLICATION writer has to keep up with new hardware developments and new
interfaces.  They have to understand and code parsers for the bytestreams
coming from each device, which (as you mentioned) are all different,
because they all support different (although overlapping) extensions.

All of these calls that currently get dumped into ioctl (because they are
not important enough to warrant an API function at the top level) suddenly
become well structure and well designed interfaces.  For example, if your
device doesn't support the terminal handling ioctls the system has to
basically figure that out and report errors on it.  With CORBA/kORBit, you
would just not implement that interface... so if someone asked for it,
they would get a null pointer to the interface, which they check for and
realize that it's impossible to do terminal stuff on.

isatty (an example of an IOCTL wrapper) suddenly becomes implicit and the
API is cleaner...

I have no problem with the "unix way", but I do think that it can be
augmented in some ways...

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/










-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


  There is a large perception of CORBA being slow, but for the most part it
  is unjustified.  I believe that the act of _designing_ a completely new

 CORBA is slow compared to some of the other solutions. The question I was 
 trying to ask is whether you should put something smaller and faster into the
 kernel space and leave CORBA in userland. 

What I'm trying to show is that CORBA itself is not slow.  CORBA can be
thought of as an IDEA, and the current implementations are
suboptimal.  For example, no CORBA implementation has been tweaked to run
well in-kernel.  Because of this, people will say that CORBA sucks and is
slow, but that's not true.

I will agree that IIOP (the standardized corba communication mechanism) is
much slower and more complex than we would want for a generalized
user-kernel communication mechanism.  The nice thing about CORBA is that
you can design your own transport to optimize things.  Thus 99% of the
time you could use a heavily optimized, very simple transport that doesn't
do much.  This is what you are asking for, and it make perfect
sense.  What CORBA buys you is the ability to use this streamlined
interface without giving up full generality...

 It's complex, it has security 
 implications surely it belongs talking something simple and fast to the kernel.

First off, there are a lot of complex systems in the kernel.  :)
Second, there are not security implications above and beyond any normal
kernel hacking.  The reason that kORBit has security implications right
now is that there _IS NO SECURITY_.  Imagine implementing sys_read with
no checks of what uid is running, and you get the idea.  

Actually security in CORBA can be done BETTER in kernel space than in user
space, but I digress...

 If you look at microkernels they talk a much simpler faster rpc protocol. 

Yes but for the most part they lose all of the advantages of CORBA as
well.  For example, every microkernel that I'm aware of can only talk to
remote servers/clients that are running the same microkernel on the remote
machine.  There are no provisions for byte swapping if the endianness is
incorrect or for having structured bytestreams (so that you can tell
things about the raw bytes coming over the line without understanding them
all).

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


  Of course.  Which is why CORBA is about putting STRUCTURE in that stream
  of random bytes coming over the wire.  Why should I have to rewrite my
  marshalling and demarshalling code every time I want to write a
  server.  read and write are fine.  But sometimes I want a
  structure.  Sometimes, my structures aren't laid out like C struct's
  either.  What then?  What if I want to send an "object" to you?
 
 Then I need to understand the object anyway. And Corba objects are horribly
 over complex. Any lisp hacker will tell you there is only one type: a list.

But alan, that's the beautiful thing.  Given a CORBA object, you can
understand its structure without knowing exactly what the contents
are.  You can effectively derive it's prototype just by inspecting it.

The only difference between lisp and a CORBA object in this respect is
that each item in the list is typed so that you have even more info about
what to do with it.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


   There is a large perception of CORBA being slow, but for the most part it
   is unjustified.  
 
 Really?  I have that same perception but I can't claim that I've measured it.
 On the other hand, I have measured the overhead of straight UDP, TCP, and
 Sun RPC ping/pong tests and you can find the code for that in any version
 of lmbench.  It should be a 5 minute task for someone who groks corba to
 do the same thing using the same framework.  If someone wants to do it,
 I'll guide them through the lmbench stuff.  It's pretty trivial, start 

Urm... thanks for the offer... but you misunderstand me if you think that
I'm claiming that kORBit is the ideal/fast implementation that everyone
has been looking for.  There is still much to be done.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


   Oh, great. So we don't have to care about formatting changes. We just
   have to care about the data changes. IOW, we are shielded from the
   results of changes that should never happen in the first place. And the
   benefit being...?
  
  What the hell are you talking about?  Did you even read my example? I was
  giving an example of extending an API, adding new functionality to it.
 
 Yes, I did. What I don't understand is how kernel mechanism for marshalling
 would make your life easier wrt changes.

I gave a very simple example of how an interface could be designed and
then later extended without breaking any user space programs.  Please
reread the example and tell me precisely what doesn't make sense, and I
would be glad to explain it better.

   Notice also that I can say ls /dev/*mouse* and get some idea of the files
   there. I can't do that for your interfaces.

  And you know what?  I can't do ls /dev/*net* and get all the network
  devices either.  Actually, one of the very cool things about CORBA is that
  you can BROWSE/LIST/SEARCH all objects currently instantiated.  Being
  able to browse all in kernel objects would be very cool.

 So what's to stop you from letting readdir() do the work? WTF do we need
 one more API for returning the list of objects?

Urm.. that was the point.  You don't add a new API (it's built into corba,
along with introspection).  Also, like I JUST MENTIONED, readdir will have
no way of knowing what is a network device in the quagmire of stuff
floating around in /dev.  (ls *net* does actually call readdir ya
know.  Perhaps you should read up more on unix... [sorry couldn't help
it. ;])

   The point being: if your program spends efforts on marshalling it would
   better _do_ something with the obtained data. And then we are back to
   the square 1.
  Uh huh, go ahead and read the example I sent to you.
 Did that.

So they you understand that no interface was broken even though more data
is now available...

   Returning to your example, I could not tee(1) the stream into file for
   later analysis. Not unless I write a special-case program for intercepting
   that stuff. I don't see why it is a good thing.

  On the contrary, it would be pretty easy to do something like that with
  CORBA.  No you wouldn't be able to use tee, but why would you want to tee
  a binary data file?  The only reason that tee works in this situation is

 For the same reasons why I use tar, gzip, whatever. I don't _want_ to
 invent a new utility every time when Joe Doe adds a new piece of interface.
 Data is data is data. I can uuencode it and send to somebody who would
 care to analize the bloody thing. Do you mean that I need to write a
 special tool for that? For _every_ member of every interface somebody
 decided to add? I don't think so. I really don't.

That's fine, I don't blame you.  Ya know what?  I even agree with you
(image that?  hehe :).  You seem to forget that a CORBA object is exactly
the same thing: a stream of bytes.  Actually it does better than
that.  It's a stream of bytes that you can introspect on and determine the
structure of.  Imaging this.  You have a bytestream flowing out of some
pipe/device/socket/file/whatever and you pipe it to a
"Decode" program.  The decode program spits this out:

sequence  
   struct { 
 float
 float
   }

This example is meant to illustrate that _in a general way_ you can decode
the structure of the bytestream.  (hence my claims about CORBA adding
structure to a standard bytestream)...  this is quite a bit more useful
than trying to figure out what 'cat' is trying to tell you about binary
data (especially if it's floating point numbers).

  line... your analysis program would have to have special purpose code to
  parse the file.  EVERY consumer of "mouse data" would have to parse the

 Or I would look at the size. Or I would say od and look at the result. Don't
 tell me what to do with the data, when I'll need to parse it I will. And
 if you expect me to bother with writing more stuff when generic tools would
 work fine - too bad, I've been there, done that and I'm not coming back.

Heh... I'm not trying to replace od or cat or tee or pipes.  What I'm
trying to do is give you a larger toolkit that gives you MORE
power.  Imagine you want to get the first float value (of each struct) of
the above stream.  Imagine a functional scripting language where you could
do this (haskell syntax stolen gratitously):

cat /dev/floatymouse | funcwith 'map fst'

Which you could then point to the "decode" type program mentioned above,
and you would now get:

sequencefloat

 BTW, you cared about size of /dev, didn't you? /usr/bin choke-full of tee-foo,
 tee-bar, yodda, yodda would be better?

You wouldn't have to replace the standard utilities.  Sorry to rain on
your parade.

  file.  That seems pretty silly to me.
 Difference between good program and bad one: the former gets used in ways
 its authors never thought about...

*cough* see above.


Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-15 Thread Chris Lattner

  For one of our demos, we ran a file server on a remote linux box (that we 
  just had a user account on), mounted it on a kORBit'ized box, and ran
  programs on SPARC Solaris that accessed the kORBit'ized linux box's file
  syscalls.  If nothing else, it's pretty nifty what you can do in little
  code...
 
 Cool!
 
 However, can you do one test for me? Do _heavy_ writes on kORBit-ized
 box. That might show you some problems. Oh, and try to eat atomic
 memory by ping -f kORBit-ized box.

I'll give that a try when I get a chance.  :)

 I've always wanted to do this: redirect /dev/dsp from one machine to
 another. (Like, I have development machine and old 386. I want all
 programs on devel machine use soundcard from 386. Can you do that?)

Yes.  Definately.  There are probably other ways of doing that... but one
of the things we implemented was a "generic" character device... and we
tested it by having a chardev server that basically reads from a
"local" (to the server) character device, and forward it over CORBA.  So
this is already implemented!  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-18 Thread Chris Lattner

cat /mnt/www/www.kernel.org/index.html
 
 can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came
 to conclusion that web filesystem is not possible... (If you can't do

Yes, if the server supports webDAV or something similar.

 listings, it is not really filesystem; you could do
 
  cat /mnt/www/www.kernel.org_index.html as well, and that's easy to
 do.) 
 
  and the CorbaFS userspace server takes care of loading the webpage and
  returning it to the kernel client.  And these new filesystems don't
  take up any extra space in the kernel, since they all talk to the same
  CorbaFS kernel module!  Not to mention being able to implement the
  filesystem in any language you like, debug the implementation in
  userspace, etc.
 
 codafs can do pretty much the same.

Yes, but codaFS is specific to filesystems.  kORBit, of course, can do
much much more, in a very uniform way.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [RFC] bloody mess with __attribute__() syntax

2007-07-05 Thread Chris Lattner

On Thu, 5 Jul 2007, Al Viro wrote:

On Thu, Jul 05, 2007 at 09:41:55AM -0700, Linus Torvalds wrote:

Note that gcc rules for __attribute__() (and that's the only source
of rules we _have_ for the damn thing) clearly say that
int __user *p;
is the same thing as
int *__user p;


Quick question: is there some reason why we have to honor the crazy gcc
rules, and cannot try to convince gcc people that they are insane?


AFAICS, they started with storage-class-like attributes.  Consider e.g.
always_inline or section; these are not qualifiers at all and you want
to have
static __attribute__((always_inline)) int foo(int *p);
interpreted with attribute applied to foo, not to its return type.


This is true, but I don't think this is related.  attributes in GCC can 
apply either to types or two decls.  In this case, the always_inline 
attribute is being applied to the decl, but other attributes could be 
applied to the return type.


-Chris

--
http://nondot.org/sabre/
http://llvm.org/
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-08 Thread Chris Lattner


This email is here to announce the availability of a port of ORBit (the
GNOME ORB) to the Linux kernel.  This ORB, named kORBit, is available from
our sourceforge web site (http://korbit.sourceforge.net/).  A kernel ORB
allows you to write kernel extensions in CORBA and have the kernel call
into them, or to call into the kernel through CORBA.  This opens the door
to a wide range of experiments/hacks:

* We can now write device drivers in perl, and let them run on the iMAC
  across the hall from you. :)
* Through the use of a LD_PRELOAD'd syscall wrapper library, you can
  forward system calls through CORBA to an arbitrary local/remote machine.
* CORBA servers are implemented as Linux kernel modules, so they may be
  dynamically loaded or unloaded from a running system at any time.  CORBA
  servers expose their IOR's through a /proc/corba filesystem.
* Filesystems may be implemented as remote CORBA objects and mounted on
  the local machine, by using 'mount -t corbafs -o IOR:... none /mnt/corba'

This are just some of the features available _RIGHT_NOW_ that are
supported by kORBit.  I'm sure that YOU can think of many more.

Implementation:
We implemented this port by providing a user->kernel mapping layer that
consists of providing standard system header files for the "user" code to
#include.  In these header files, we do the mapping required.  For
example, we implement a  that #defines printf to printk (as a
trivial example).  Only user level code sees or uses these wrappers... all
of our modifications to the Linux kernel are contained within the
linux/net/korbit subdirectory.  

This is currently implemented with a 2.4.0test10 kernel, although forward
porting should be very easy.  This project was implemented as a cs423
semester project by Chris Lattner, Fredrik Vraalsen, Andy Reitz, and Keith
Wessel at the University of Illinois @ Urbana Champaign.

Unresolved issues:
* Our poll model is not optimial.  Currently we actually do a real poll on
  a (struct socket *) set.  This causes relatively high latencies (on the
  order 1 second, worst case) for CORBA requests.  Our waitqueues are not
  working quite as well as they should.  :)
* Security is completely unimplemented.  Someone could use corba
  interfaces to read any file on your system, for example (if the
  CORBA-FileServer module is installed).  Thus, this is really more for
  prototyping and development than actual real world use. :)

If you have any questions or comments, please feel free to contact us at:

Chris Lattner, Fredrik Vraalsen, Andy Reitz, Keith Wessel
<[EMAIL PROTECTED]>

btw, yes we are quite crazy, but what good is it to be normal and
conformist afterall?  :)




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-12 Thread Chris Lattner


On Sat, 9 Dec 2000, Mohammad A. Haque wrote:

> It was just an example. Basically, you'd be able to do in with just
> about any language that has ORBit bindings.
> 
> Ben Ford wrote:
> > Why would you *ever* want to write a device driver in perl???
> 

Precisely... but also, there could be a case where perl would make
sense.  Consider an FTP filesystem.  There performance is not dictated by
the speed of the language, it's limited by bandwidth.  It could make sense
to write your almighty FTPfs like this:

1. Prototype it in Perl, get all the bugs out.
2. Rewrite in C in userspace, get all the bugs out.
3. recompile/relink in kernel space with no source modifications
4. ship product.  :)

The great thing that kORBit buys you is insulation of kernel space from
the drivers that are running... kinda like a microkernel.  I'm not going
to start a flamewar here about Linux and microkernels, but when doing
initial development work for a driver, the test/crash/reboot/fsck cycle is
a real pain (okay, maybe journalling helps a little, but you get the
idea).  What we're offering goes more like this:

1. Boot kernel
2. Install corbafs module for example
3. Start test filesystem in user space
4. mount test user space filesystem
5. test it, oh crap, it segfaulted.
6. CorbaFS gets exceptions trying to communicate to server, which it
relays to the kernel as -errno conditions.
7. You safely unmount corbafs
8. fix your bug
9. goto step #2.

Which is arguably nicer.  :)

The whole idea is that a bastard driver shouldn't take your kernel down if
you know it to be unreliable... if you trust the driver, then by all
means, don't use kORBit.  Also, kORBit is useful when you don't WANT
something in the kernel... and I won't bring up the whole user level
filesystem debate again... :)

-Chris

http://www.nondot.org/~sabre/os/
http://korbit.sourceforge.net/

> -- 
> 
> =
> Mohammad A. Haque  http://www.haque.net/ 
>[EMAIL PROTECTED]
> 
>   "Alcohol and calculus don't mix. Project Lead
>Don't drink and derive." --Unknown  http://wm.themes.org/
>[EMAIL PROTECTED]
> =
> 


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> > > It was just an example. Basically, you'd be able to do in with just
> > > about any language that has ORBit bindings.

> Agree.  I remember a big complaint about Windows was the huge APIs,
> compared with Unix' tiny list of syscalls.  And then I saw the GNOME
> docs... ew!

Err... how about this:  Give me two or three kORBit syscalls and I can get
rid of all the other 100+ syscalls!  :) 

We're not trying to port GNOME to the kernel.  We're not trying to bloat
the kernel API, actually we're trying to do the opposite.  Consider this:

1. kORBit adds about 150k of code to the 2.4t10 kernel.
2. kNFS adds about 100k of code to the 2.4t10 kernel.
3. kORBit can do everything kNFS does, plus a WHOLE lot more: For example
   implement an NFS like server that uses SSL to send files and
   requests... so it is really actually "secure".

> > Yeah... "Infinitely extendable API" and all such. Roughly translated
> > as "we can't live without API bloat". Frankly, judging by the GNOME


Whoa, you are so right... a well defined API is a terrible thing!
Instead, lets try doing the following:

1. Define our API to be a bizarre and poorly documented collection of
   inline functions, macros, and normal functions.
2. Lets make this API have all kinds of subtle side effects, and lets not
   document these!
3. Lets raise the level that people have to be at to even TOUCH our
   APIs.  Only uber-hackers that have worked with the system for 1+ years
   should be able to comprehend what's going on in the big picture...
4. Nobody that is "allowed" to work on this thing is dumb enough to
   produce any bugs.  Because of that, debuggers are outlawed and anyone who
   attempts to debug code is shunned from existance...
5. Lets duplicate code all over the place that does mostly the same thing,
   but has subtle differences.
6. Lets structure the file layout such that files are in poorly grouped
   catagories, often with ambiguous locations, and lets drop HUNDREDS of
   .c files into individual directories.  After all that's much easier
   than actually sorting them out. 

oh wait, we have this, the Linux kernel.


Yes yes, this was definately a pot shot at the kernel tree, and I'm not at
all serious about this.  The fact of the matter, however, is that there
are a lot of cool things that a well defined interface for kernel
programming can buy you.  Yes, there is an overhead, but do you really
care about how performant your parrellel port scanner driver is?  I'm not
advocating that people write their fibre channel drivers with kORBit. :)

Actually, if you really want to get technical about this thing, I never
said anyone should use this at all.  :)  I just mentioned that it is out
there and if people feel like it they can use it or play with it.  I
happen to think it addresses a point that is currently poorly addressed in
the Linux kernel: you can't do RPCs to the kernel in a well defined
manner.  Linux is facing a crisis that is growing every day, and that is
that there is no well defined interface for drivers to use... so
compatibilty across releases it almost impossible.

Don't worry about kORBit.  Like most open source projects, it will simply
die out after a while, because people don't find it interesting and there
is really no place for it.  If it becomes useful, mature, and refined,
however, it could be a very powerful tool for a large class of problems
(like moving code OUT of the kernel).

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> > Don't worry about kORBit.  Like most open source projects, it will simply
> > die out after a while, because people don't find it interesting and there
> > is really no place for it.  If it becomes useful, mature, and refined,
> > however, it could be a very powerful tool for a large class of problems
> > (like moving code OUT of the kernel).
> 
> I do have one sensible question. Given that corba is while flexible a 
> relatively expensive encoding system, wouldn't it be better to keep corba
> out of kernel space and talk something which is a simple and cleaner encoding

Very good point.  I think this distills down to common misconception of
CORBA: that it must be slow.  Actually, a LOT of research has been done
to improve CORBA performance, but you have to keep in mind one very
IMPORTANT THING:

CORBA does NOT dictate a transport or data format for the data going out
"over the wire".

CORBA does specify one "standard" used for interoperability: GIOP.  It
also specifies one important incarnation of GIOP: IIOP (GIOP over
IP).  kORBit currently uses this to communicate from user space to kernel
space... obviously this is not ideal.  :)

There is absolutely NOTHING that prevents us from defining our own "data
marshalling" protocol/algorithm that is lighter weight than IIOP.  In
fact, we had planned to implement that for our project, but of course, ran
out of time.  It would be relatively straight forward to define this
extension in the ORBit code base, and in fact "ORBit 2" is striving to
make this much much easier than it is now (basically sorting out IIOP
specific code from the main code base better, and generally cleaning
things up).  Currently, ORBit supports IIOP and its own proprietary (as
far as I know) Unix domain socket protocol (obviously only works
intramachine).

There is a large perception of CORBA being slow, but for the most part it
is unjustified.  I believe that the act of _designing_ a completely new
protocol, standardizing it, and making it actually work would be a huge
process that would basically reinvent CORBA (obviously some of the design
decisions could be made differently, but all the same issues would have be
dealt with).

CORBA, today, gives us superior interoperability (through IIOP), with
extensibility for the future.  As Alexander Viro mentions, 9P may be a
better protocol for local communications... so CORBA gives us a framework
to try that out with.  Even IIOP is smart enough to, for example, only do
endian swapping of data if the two machines actually differ... so at least
it doesn't specify a "network" byte order. 

CORBA isn't ideal for all applications, it certainly isn't the ideal IPC
or RPC mechanism (the L4 designers would have a laughing fit if you tried
to use it for _general_ IPC), but it is very useful for a lot of
things.  kORBit is very pre-alpha code, it works, but has lots of room for
enhancements, experiementation, and toying around with.  

That's what makes it cool.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> > Err... how about this:  Give me two or three kORBit syscalls and I can get
> > rid of all the other 100+ syscalls!  :) 

> Like it ioctl() does it? Number of entry points is _not_ an issue. Diversity
> of the API is. Technically, kernel has 1 (_o_n_e_) entry point as far as
> userland is concerned. int 0x80 on x86. Can't beat that, can you?

Err shame on you, don't forget about lcall and exceptions, and interrupts,
and... That is technically more than _o_n_e_ "entry point".  :)  Oh wait,
what about sysenter/exit too? :)

No I can't beat that.  But if you look at the hack job of a system call
table we have, you can see that there is no _really_ standard way of
passing parameters.  Oh sure, most of the time, stuff is passed in
registers.  Sometimes we get a pointer to an argument struct.  Because of
this wonderful design we get all kinds of stuff like sys_oldumount vs
sys_umount and others...

> Yes, standard RPC mechanism would be nice. No, CORBA is not a good candidate -
> too baroque and actually known to lead to extremely tasteless APIs being
> implemented over it. Yes, I mean GNOME. So sue me.

Hrm... because I'm stupid, please explain how CORBA is too baroque...  I
have no problem with you not liking GNOME... you're a kernel hacker, so
you're not supposed to like GUI's.  :)  [just kidding!!!] CORBA doesn't
preclude nasty APIs any more than C does.  It also doesn't preclude *nice*
APIs that are upgradable and extensible in the future (and that means
without breaking backwards compatibility).  Please don't tell me that OOP
is bad... or else we will have the eviscerate the VFS layer from the
kernel (amount other subsystems)... :)

> I would take 9P over that any day, thank you very much.

Like I mentioned in a previous email, CORBA does not preclude 9P.  What
it does buy you though, is compatibility with LOTS of preexisting CORBA
tools.  How much development infrastructure is there for 9P?  I thought
so.  :)

For one of our demos, we ran a file server on a remote linux box (that we 
just had a user account on), mounted it on a kORBit'ized box, and ran
programs on SPARC Solaris that accessed the kORBit'ized linux box's file
syscalls.  If nothing else, it's pretty nifty what you can do in little
code...

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> > I do have one sensible question. Given that corba is while flexible a 
> > relatively expensive encoding system, wouldn't it be better to keep corba
> > out of kernel space and talk something which is a simple and cleaner encoding

> p9fs exists.  I didn't see these patches since August, but probably I can poke
> Roman into porting it to the current tree.  9P is quite simple and unlike
> CORBA it had been designed for taking kernel stuff to userland.  Besides,
> authors definitely understand UNIX...

One thing that you might want to mention Alexander: 9P is not a general
communications protocol.  In fact, it doesn't work very well across the
internet at all.  To get decent performance, the Plan9 group (which, is a
very cool group.  :) has to specify a new protocol that competes with TCP
on the level of complexity (IL: http://plan9.bell-labs.com/sys/doc/il/il.html)

Also, 9P is a general communications framework only in the context of
Plan9 itself.  In reality it only applys directly/well to filesystem
related issues... the reason it works well in Plan9 is that _everything_
is a file (part of the beauty of plan9).  

With some elbow grease, 9P could probably be made to work in the kORBit
framework.  It's not even that big of a deal: it just takes time.  Believe
me when I say that IIOP is not a very good user->kernel communications
mechanism.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


/me trims down CC list...

>   Local? Funny. It lives atop of TCP or IL quite fine. What's
> even funnier, I can use it to export /proc from CPU server to workstation
> and use _that_ for remote debugging. Ditto for window system. Ditto for
> DNS. Ditto for plumber. No, not on Linux...

No not on linux.  No definately not on Windows, no not on the BSD's
either.  Oops, wasn't interoperability an important part of the Linux
kernel design?  Didn't we want to use and follow and define _real_
standards? 

>   What you do is mapping your RPC to hierarchical namespace. After
> that... What extensibility do you need? You can use every utility written
> for core UNIX API (open()/read()/write()/close()) as a client. No need
> to reinvent the wheel...

Heh, that's interesting.  _getting_ the data isn't hard in 9P, actually
using it or displaying it in a meaningful way commonly requires something
more than cat.  :)

The overriding problem with a simplistic 9P port is that linux kernel
functionality is not very well modularized.  In fact, this prevents
wonderful CORBA integration as well: An agressive filesystem (for
example) often likes to touch the buffer cache or even the VM layer
(*cough*) directly.  Some don't use the prescribed interfaces, because
nothing enforces it.  9P works wonderfully for Plan9 because they had the
luxery of redefining/rewriting the whole OS, and the whole interaction
with user space processes.  We, unfortunately, don't have that
possibility.  :)

Please don't get me wrong.  I'm not opposed to other ideas, and 9P may in
fact turn out to be a very nice protocol that would be able to support 
much kORBit level functionality).  I do maintain that by writing a custom
user->kernel marshalling library, you could obtain better peformance than
9P though, because you could take advantage of lots of machine specific
optimizations.  Hell you could even pass things in register if you'd have
<= 4 args.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> > Err shame on you, don't forget about lcall and exceptions, and interrupts,
> > and... That is technically more than _o_n_e_ "entry point".  :)  Oh wait,
> > what about sysenter/exit too? :)
> OK, you got me on lcall (however, that's iBCS-only, IIRC), but the rest...
> what the hell does userland to interrupts?  OK, make it 2 - pagefault
> can be arguably used in that way.

The reason that I considered exceptions and interrupts is that often,
exceptions get reflected as signals to the running processes (SIGSEGV,
SIGFPE, SIGILL, others?), and interrupts can wake up processes (from
sys_poll among others).  I was considering more of the user->kernel and
kernel->user transitions... anyways, that's really besides the point.  :)

> > this wonderful design we get all kinds of stuff like sys_oldumount vs
> > sys_umount and others...
> Check how often anything uses the majority of that stuff...

Correct, it's for backwards compatibility with old programs (for example
libc5 uses a lot of those "old" syscalls).  

> > > Yes, standard RPC mechanism would be nice. No, CORBA is not a good
> candidate - > > too baroque and actually known to lead to extremely
> tasteless APIs being > > implemented over it. Yes, I mean GNOME. So

> Check 9P and compare. Really. Section 5 of Plan 9 manpages. Available on
> plan-9.bell-labs.com/sys/man/

That's fine.  Since the server is down (or the URL is bad), can you please
give me an example of how 9P is better than CORBA?  I freely admit to not
knowing much about 9P... how much do you know about CORBA (aside from
your opinion that GNOME uses it, and therefore it is bad. ;)?

> > without breaking backwards compatibility).  Please don't tell me that OOP
> > is bad... or else we will have the eviscerate the VFS layer from the
> > kernel (amount other subsystems)... :)

> OOP is a nice tool. However, it's a tool that has incredible potential of
> shooting one's foot. It's wonderful if you have sane set of methods. And
> that's a _big_ if. "Easily extensible" is not an absolutely good thing -
> C++ wankers all over the world are busily proving it every day. Heck, they
> make a living out of that. IOW, the problem with interface changes is _not_
> in converting the old code. It's in choosing the right changes. And that
> part of the game can't be simplified.

Oif.  That's like telling someone that C is evil because it has for loops,
and for loops can be used to write nasty code.  "just write in
assembler" he says.  :)  I would claim that someone could write a bad
program (or shoot themselves in the foot) with any turing complete
language.  C++ definately give you more rope to do that with, but used
wisely, it can also be nice.  The trick is to just not have to work with
other peoples C++ code.  :)  Hey, did I mention that kORBit and all its
extensions are written in C? :)

> > Like I mentioned in a previous email, CORBA does not preclude 9P.  What
> > it does buy you though, is compatibility with LOTS of preexisting CORBA
> > tools.  How much development infrastructure is there for 9P?  I thought
> > so.  :)

> All UNIX userland on the client side. lib9p on the server side (23Kb of sparse
> C). Examples of use in servers - see the aforementioned site.

Err... yeah, so you're effectively mapping UNIX/POSIX across 9P.  That's
not very creative, and you could do the same thing with CORBA.  I ask
again, "How much development infrastructure is there for 9P?".  If you say
"just use unix", then what is the point of 9P at all?  (on linux).  Linux
already has most of posix (and some would claim all of the "good
stuff" in posix.).

> > For one of our demos, we ran a file server on a remote linux box (that we 
> > just had a user account on), mounted it on a kORBit'ized box, and ran
> > programs on SPARC Solaris that accessed the kORBit'ized linux box's file
> > syscalls.  If nothing else, it's pretty nifty what you can do in little
> > code...
> Duh. And what's new about that?

The "new" part is that our servers were < 100 lines of code each.  Compare
that to kNFS.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> > either.  Oops, wasn't interoperability an important part of the Linux
> > kernel design?  Didn't we want to use and follow and define _real_
> > standards? 
> Erm... 9P stub exists for Linux. It exists for FreeBSD. I suspect that
> it exists for other *BSD too - never checked that.

Okay, so there are _stubs_ for these platforms.  How many languages are
there bindings for?  What about Win32 (gross I know, but at least for now,
it is important)?  What about OS/2 what about Amiga and palm?  CORBA is
very popular for certain things... even inferno isn't nearly as big...

> > Heh, that's interesting.  _getting_ the data isn't hard in 9P, actually
> > using it or displaying it in a meaningful way commonly requires something
> > more than cat.  :)
> Same for any other RPC mechanism.

Of course.  Which is why CORBA is about putting STRUCTURE in that stream
of random bytes coming over the wire.  Why should I have to rewrite my
marshalling and demarshalling code every time I want to write a
server.  read and write are fine.  But sometimes I want a
structure.  Sometimes, my structures aren't laid out like C struct's
either.  What then?  What if I want to send an "object" to you?

> > nothing enforces it.  9P works wonderfully for Plan9 because they had the
> > luxery of redefining/rewriting the whole OS, and the whole interaction
> > with user space processes.  We, unfortunately, don't have that
> > possibility.  :)
> Notice that they _don't_ export random internal APIs to userland.

And neither do we.  Beautiful isn't it?  :)

> > Please don't get me wrong.  I'm not opposed to other ideas, and 9P may in
> > fact turn out to be a very nice protocol that would be able to support 
> > much kORBit level functionality).  I do maintain that by writing a custom
> > user->kernel marshalling library, you could obtain better peformance than
> > 9P though, because you could take advantage of lots of machine specific
> > optimizations.  Hell you could even pass things in register if you'd have
> > <= 4 args.  :)
> 
> You mentioned OOP, didn't you? Encapsulation is a good thing and what you
> are talking about is "layering violations made Real Easy(tm)".
> I simply don't see why _that_ is a good goal.

I completely fail to see how I'm breaking encapsulation or making layering
violations easy.  Why I'm talking about is the fact that CORBA doesn't
care how you ship bytes around: it's a higher level protocol that cares
about shipping "things" around.  Bytes are old school, "stuff" is the wave
of the future.  ;)  heh.

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner

> > > plan-9.bell-labs.com/sys/man/
> Arrgh. s/plan-9/plan9/. My apologies.

Cool, thanks, will read.  :)

> IDGI. What 9P gives is an RPC mechanism that uses normal (as in "named streams
> of characters") representation on the client side and very light-weight
> library on the server side. It looks like you are trying to do a mechanism
> that would export arbitrary _internal_ kernel APIs. I really don't see
> what you would want it for.

Which is exactly why it doesn't work well for many applications.  The
problem is this: how do you get from a byte stream to a structured data
stream?  There are many answers:

1. Keep your data structures so simple, that it's obvious.  Not a good
choice.  :)
2. Defined interfaces to the bytestream for every interface that you
define.
3. Define a standard for inflating/deflating "things" into
bytestreams.  Oh wait, that's what corba does.  :)

The point of the matter is that CORBA gives you a nice, clean, well
supported method of not having to write marshalling/demarshalling code
every time you want to use an interface.  Not only that, it lets you
define "fast paths" for special cases (ie, with user level corba, if you
are communicating between two objects in the same processes, you can do
direct calls and do no marshalling).  These fast paths do not interfere
with the generality of the system at all.

The 9P way of doing things is not fundementally new, they just applied the
idea that "everything is a file" more broadly.  What annoys me is that it
is not immediately obvious how to "demarshall" the data that comes out of
/dev/mouse for example.  Combine that with the problem that /dev/mouse
might change format in the future (okay stupid example, but you get the
idea) to use floating point coordinates, and things certainly get hairy.

Why reinvent the wheel countless times?

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: CORBA vs 9P

2000-12-13 Thread Chris Lattner


> > Okay, so there are _stubs_ for these platforms.  How many languages are
> > there bindings for?
> Grr... Let's define the terms, OK? What is available: kernel code that
> represents the client side of RPC as a filesystem. Userland clients do
> not know (or care) about the mechanisms involved.

But they DO CARE about the format of the data.

>   And files with structure are things of dreadful past. BTDT.
> You really need to... work with an OS that would have and enforce
> "structured files"  to appreciate the beauty of ASCII streams.

Ahhh, so ASCII streams are a wonderful thing.  Are you an XML fan?  :)

>   However, that's a different story. What I _really_ don't understand
> is the need to export anything structured from kernel to userland.

Okay, how about a few examples.  How about /proc/meminfo?  How about the
"stat" structure?  How about /proc/stat?  You seem to be indicating that
ASCII files are fine for general exportation of kernel information.  The
/proc filesystem begs to differ.  One specific example is the
/proc/meminfo file.  Why is it that one field is 0 now?  Ouch we can't
change the format of the file because we'll break some program.  Crap, you
want to add a field, well, tough luck.

The struct stat example is one _trivial_ example of "the need to export
anything structured from kernel to userland".

>   IOW, I would really like to see a description of use of your
> mechanism. If it's something along the lines of "let's take a network
> card driver, implement it in userland and preserve the current API" -
> see the comment about layering violations. You've taken an internal
> API and exposed it to userland in all gory details. See also your own
> comment about internal APIs being not convenient for such operations.

I'm not trying to dictate interfaces.  I'm not trying to tell people what
to use this stuff for.  I'm arguing that it's useful and that you can do
very interesting things with it.

> If it's something else - I wonder what kind of objects you are talking
> about and why opaque stuff (== file descriptors) would not be sufficient.

Opaque stuff is fine.  I have no problem with file descriptors.  They
effectively solve the exact same class of problems that CORBA does, except
that they add significant _API BLOAT_ because every little "method" that
implements them gets a syscall.  Oh yeah, they you get ioctl
too.  Funness.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> OK, now I'm completely confused. 
>   * which complex data structures do you want to export from the kernel
> in non-opaque way? 
>   * which of those structures are guaranteed to remain unchanged?
>   * if you have userland-to-userland RPC in mind - why put anything
> marshalling-related into the kernel?

Okay, I think I did my best to completely confuse you.  :)  #1: CORBA
objects _are_ opaque.  #2 is irrelevant due to #1.   #3: userland->userland 
is not the interesting part.  We want kernel->userland or user->kernel, as
the common (ie fast) case, but we also want to do client->client or
kernel->kernel without anything breaking.

> > /dev/mouse for example.  Combine that with the problem that /dev/mouse
> > might change format in the future (okay stupid example, but you get the
> > idea) to use floating point coordinates, and things certainly get hairy.

> HUH? OK, suppose it had happened. Do you really expect that you will not
> need to change your applications? I mean, if you expected a bunch of

YES.  You should not have to change your apps at all.

> ints and received a bunch of doubles... You either need to decide on
> rounding (and it's a non-obvious question) or you need to change quite a
> bit of code in your program. It goes way past the demarshalling, no
> matter whether you use CORBA, 9P or printf/scanf.

NO.  You want leagacy program to "just get" rounded ints, and new programs
to get the "full precision" of the floating point #'s.

> OK, suppose you have a CORBA-based system and mouse drivers' API had been
> changed - they really want to return floating point coordinates. How will
> CORBA help you? Aside of making your programs scream aloud, that is.

Okay, maybe this isn't a terrible example afterall.  :)  Consider this
theoretical API for a mouse, that I'm certainly not saying is wonderful or
perfect, just consider it to be an example (on the fly), and no it
doesn't let you poll for mouse activity, this is just off the cuff:

Original interface for the mouse:

interface mouse {
 void getPosition(out long X, out long Y);
};

So if you get the mouse interface in CORBA (which is an opaque object),
you get back something that you can make requests to, and, for example,
get the coordinate the mouse is currently at.  Okay, that's fine, 15 years
later and after much legacy software has been developed, company X
develops a new high precions floating point mouse.  Well crap, don't want
to change the interface.  Now instead of getting that, you get the
"mouse2" or "floaty mouse" interface (for sake of argument suppose that
we actually did want to use FP arithmetic in the kernel, that's an
artifact of a bad example, not a bad point :):

interface floatymouse : extends mouse {
  void getFloatyPosition(out float X, out float Y);
}

Now people that get the "/dev/mouse" interface get a floatymouse.  Does
this break all that leagacy mouse wielding code?  No, because we used
inheritance, the floatymouse is a superset of the mouse, and the original
interface still works.  The server/kernel side of the floaty mouse just
chops off the decimal places when you use the getPosition method.  

This is the kind of stuff that I'm talking about.  The concept that a
mouse is more than a byte stream... that there is actual structure that
must be understood before a datastream can be used.

Yes, I am very aware that there are other ways of doing this, but CORBA is
a very general & powerful system that is also quite mature (unlike
kORBit.  ;)

I hope this helps clarify what the heck I'm talking about.  :)

-Chris

btw, just imagine how much cleaner the mount interface could be... ;)

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: CORBA vs 9P

2000-12-13 Thread Chris Lattner


I think that I addressed most if not all of this email in my previous
one... let me know if I missed something.

-Chris

btw, thanks for putting up with me, I know I can be obstinate
sometimes. :)  

On Wed, 13 Dec 2000, Alexander Viro wrote:

> 
> 
> On Wed, 13 Dec 2000, Chris Lattner wrote:
> 
> > > > Okay, so there are _stubs_ for these platforms.  How many languages are
> > > > there bindings for?
> > > Grr... Let's define the terms, OK? What is available: kernel code that
> > > represents the client side of RPC as a filesystem. Userland clients do
> > > not know (or care) about the mechanisms involved.
> > 
> > But they DO CARE about the format of the data.
> 
> And how would CORBA help here? Because format changes are usually coming
> from the _contents_ changes. And if you don't care about the contents -
> why the hell do you retrieve the object int the first place?
> 
> > >   And files with structure are things of dreadful past. BTDT.
> > > You really need to... work with an OS that would have and enforce
> > > "structured files"  to appreciate the beauty of ASCII streams.
> > 
> > Ahhh, so ASCII streams are a wonderful thing.  Are you an XML fan?  :)
> 
> No, thanks.
> 
> > >   However, that's a different story. What I _really_ don't understand
> > > is the need to export anything structured from kernel to userland.
> > 
> > Okay, how about a few examples.  How about /proc/meminfo?  How about the
> > "stat" structure?  How about /proc/stat?  You seem to be indicating that
> > ASCII files are fine for general exportation of kernel information.  The
> 
> Yes, _if_ you take care to think what you are exporting. /proc/meminfo is
> a shi..ning example of _not_ doing that over many years.
> 
> > /proc filesystem begs to differ.  One specific example is the
> > /proc/meminfo file.  Why is it that one field is 0 now?  Ouch we can't
> > change the format of the file because we'll break some program.  Crap, you
> > want to add a field, well, tough luck.
> 
> Oh, cool. So CORBA would magically change the definition of the structure in
> your (C/Modula-3/APL/COBOL) programs. How? And what would happen with the
> code that used to access the field in question?
> 
> > The struct stat example is one _trivial_ example of "the need to export
> > anything structured from kernel to userland".
> 
> It's a trivial example of "why you need to think before deciding what to
> export".
>  
> > >   IOW, I would really like to see a description of use of your
> > > mechanism. If it's something along the lines of "let's take a network
> > > card driver, implement it in userland and preserve the current API" -
> > > see the comment about layering violations. You've taken an internal
> ^
> > > API and exposed it to userland in all gory details. See also your own
> > > comment about internal APIs being not convenient for such operations.
> > 
> > I'm not trying to dictate interfaces.  I'm not trying to tell people what
> > to use this stuff for.  I'm arguing that it's useful and that you can do
> > very interesting things with it.
> 
> And when interface changes, you do what, exactly?
> 
> > > If it's something else - I wonder what kind of objects you are talking
> > > about and why opaque stuff (== file descriptors) would not be sufficient.
> > 
> > Opaque stuff is fine.  I have no problem with file descriptors.  They
> > effectively solve the exact same class of problems that CORBA does, except
> > that they add significant _API BLOAT_ because every little "method" that
> > implements them gets a syscall.
> 
> Huh? Could you elaborate, please?
> 
> 

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-13 Thread Chris Lattner


> > NO.  You want leagacy program to "just get" rounded ints, and new programs
> > to get the "full precision" of the floating point #'s.

> What rounded ints? Rounded to zero? To nearest integer? To plus or minus
> infinity? Does program have something to say here?

The exact same thing that older mouse drivers did.  In this particular
case, it doesn't matter anyways, because it's a simple example.  If you
REALLY care about rounding those coordinates a certain way, it's trivial
to add an extra component to the interface.

> ...
> My mouse example
> ...

> Oh, great. So we don't have to care about formatting changes. We just
> have to care about the data changes. IOW, we are shielded from the
> results of changes that should never happen in the first place. And the
> benefit being...?

What the hell are you talking about?  Did you even read my example? I was
giving an example of extending an API, adding new functionality to it.

> Notice that we could equally well add /dev/floatymouse and everything that
> worked with old API would keep working. The only programs that would need
> to know about floats would be ones that would... need to know about floats
> since they want to work with them.

Wonderful solution.  This is exactly what you would _have_ to do.  Are you
aware of why devfs came into existence?  It is because there are already
so damn many device nodes in /dev.  Lets keep adding more.  

Why is it (for example) that on my machine (mandrake 7 install, no devfs),
I automatically get inodes for ISDN & PPP!?!  Well, because ISDN has a
slightly different interface, and you're talking to a different driver
that is not transparently hidden to the user.  Why can't I just open
/dev/net0 and get the first network device?  Because we have so many
inconsistent, poorly design, inextensible interfaces laying around, thats
why.

> Notice also that I can say ls /dev/*mouse* and get some idea of the files
> there. I can't do that for your interfaces.

And you know what?  I can't do ls /dev/*net* and get all the network
devices either.  Actually, one of the very cool things about CORBA is that
you can BROWSE/LIST/SEARCH all objects currently instantiated.  Being
able to browse all in kernel objects would be very cool.

> The point being: if your program spends efforts on marshalling it would
> better _do_ something with the obtained data. And then we are back to
> the square 1.

Uh huh, go ahead and read the example I sent to you.

> Returning to your example, I could not tee(1) the stream into file for
> later analysis. Not unless I write a special-case program for intercepting
> that stuff. I don't see why it is a good thing.

On the contrary, it would be pretty easy to do something like that with
CORBA.  No you wouldn't be able to use tee, but why would you want to tee
a binary data file?  The only reason that tee works in this situation is
because it's agnostic to the format of the data coming off the
line... your analysis program would have to have special purpose code to
parse the file.  EVERY consumer of "mouse data" would have to parse the
file.  That seems pretty silly to me.

> I also don't see where the need in new system calls (or ioctls, same shit)
> comes from. Notice that your way is much closer to new system call than
> read()/write() of the right stuff.

A new syscall was one example.  It would be very simple to implement this
by making _yet_another_ device node in /dev and issue reads and writes to
it.  That's more of a syntactic issue than a symantic one.

> As for the proc/meminfo... What would you do to a userland programmer who
> had defined a structure like that? Let's see: way too large, ugly as hell,
> many fields are almost guaranteed to become meaningless at later point...
> It was not designed, it got accreted. And that's very mild description -
> judging by results it might as well be s/ac/ex/.

There is no provision in CORBA that prevents the creation of ugly
interfaces.  There is no provision in C that prevents them either, as
witnessed by this example.  At least with CORBA, I can extend the
interface to actually return meaningful results while still maintaining
backwards compatibility with programs that don't actually look at that
field (as is done now by sticking the 0 in there).

The trick is having enough foresight to design robust interfaces.  I'm not
saying that I'm an expert at that, and obviously nobody can be.  What I
_am_ saying is that CORBA/kORBit gives you flexibility in how you do
interfaces, and it lets you "future proof" them (obviously only within
reason).  Adding a new device node everytime an API gets changed is like
defining a different API for every hard disk that exists.  Seems kinda
silly.

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at 

Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner

> > 1. kORBit adds about 150k of code to the 2.4t10 kernel.
> > 2. kNFS adds about 100k of code to the 2.4t10 kernel.

> So can you implement a kNFS server in kORBit that takes
> less than 50kB of RAM?  Otherwise it's still a contributor
> to bloat and this argument won't work ;)

Actually the kORBitNFS server would have to take -50K of code to break
even.  :)  The point was that kORBit lets you do a lot more... so
hopefully that 50k of generality gives you something.  :)

> I guess it's time to stop the flaming and to see what can
> be achieved using kORBit. The people who favour kORBit should
> IMHO be left alone and given the opportunity to show what can
> be achieved with kORBit ... if they don't achieve anything,
> the nay-sayers can always claim their "victory"; if something
> useful comes out the kORBit people can claim usefulness.

Agreed!

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ORBit speed measure

2000-12-14 Thread Chris Lattner


>> There is a large perception of CORBA being slow, but for the most part
>> it is unjustified.  

> Well, I've measured using function calls through ORBit is 300 times
> slower than using dynamic loading.
> ...
> Which gives me the dl is about 333 times faster than ORBit.

You leave so many details out, and you sortof miss the point with my
statement.  The idea is that CORBA doesn't specify a transport to use _at
all_ except when doing inter-orb communication (which you basically have
to use IIOP to be compatible).  This is why, for example, if you
ior-decode ORBit IORs that they contain two parts: an IIOP portion, and a
Unix domain socket portion.  Basically ORBit is optimizing the
intramachine case, because it is very common for purposes that ORBit is
mainly used: GNOME.

You didn't specify, but I assume that you were using the Unix sockets
transport for your benchmark.  I am also assuming that the two things
communicating were in different processes.  If you wanted to make a more
fair test case, you would have to use some form of IPC mechanism to
communicate between two distinct processes.  Comparing a function call to
an RPC mechanism will never look good.  :)

There are many other optimizations that one can make the transport faster
that ORBit doesn't implement.  For example, you could mmap (shared) data
buffers between the two processes communicating (of course, you still need
to wake processes up, which is why it hasn't been done yet), or you could
use pipes (if you can control forkness) or any number of things.  If you
are communicating with a server in the same process as your client
(because it was dynamically loaded) you can even do a direct function
call.

When I said that "for the most part it is unjustified" to think that CORBA
is slow, what I meant is that CORBA isn't slow: implementations are.  Just
look at the difference between ORBit and all those other orbs out
there... orders of magnitude speed differences, with little or no
functionality differences.

What CORBA gives you is the power to have optimizations for important
special cases (in kORBit's case, one of those would be User->kernel),
WITHOUT sacrificing the big picture or the ability to be interoperable...

I am not claiming that kORBit is where it needs to be right now.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


> >that is not transparently hidden to the user.  Why can't I just open
> >/dev/net0 and get the first network device?  Because we have so many
> >inconsistent, poorly design, inextensible interfaces laying around, thats
> >why.

> This is a bad example, but a (perhaps?) good point.  It seems it should be 
> possible to implement an unlimited number of TCP/IP devices, each 

Part of my argument is that although every kernel interface boils down to
something as simple as ioctl (where you have lots of api's multiplexed
onto one data sink, int 0x80 is another example, kORBit is another), some
ways of doing this multiplexing are better than others.  I look at it as a
hierarchy:

1. CORBA/kORBit
2. int 80/lcall
3. ioctl

Each level is more structured than those below it.  Each interface is also
"cleaner" than those below it.  I don't think that anyone would argue that
we should replace int 80 and friends with an interface in the spirit of
ioctl (even though it would be functionally identical).

> representing a connection, and each connected, disconnected, etc. by 
> ioctls...  Of course, I could be way wrong about this

ioctl's... they come up again.  Everytime that I look at the beautiful,
clean, abstract interface that Unix exposes... it makes me happy.  But
unfortunately, if you look closer, you realize that the Unix API isn't
really clean or nice at all... there is this dumping ground for odds and
ends that don't fit into the standard model.  So when you show someone the
standard model, it looks clean and pretty... but someone always forgets to
mention ioctl.

I would claim that ioctl is one of the biggest reasons that kORBit needs
to exist.  See below.

> >I can't do ls /dev/*net* and get all the network
> >devices either.  Actually, one of the very cool things about CORBA is that

> Different network devices are _very_ different...  You can't broadcast on a 
> modem, for instance, nor can you use promiscuous mode on a modem... You 
> also can't do byte-by-byte raw mode on an ethernet...

Precisely.  But they do all have very common interfaces.  For example, the
standard network interface would probably have "transmit/receive
block" and "getstatus" commands.  This does not mean that they cannot
implement other interfaces, however.

Imagine this situation for writing your packet sniffer:

1. You open up /dev/net0 and get the Network interface
2. You query the network interface to see if it has an implementation
   of the promiscuous interface.
3. If not, you bail, because it doesn't support it.
4. If, you go ahead and use it.

Contrast that to:

1. Survey all the different interfaces that are known (at development time
   of course) to have promiscuous interfaces.
2. Find out which ones are active.
3. Depending on what kind of interface it is, load a library that can
   understand the byte format comming off the device.
4. Use that library.

The problems with the second (currently used) approach is that the
APPLICATION writer has to keep up with new hardware developments and new
interfaces.  They have to understand and code parsers for the bytestreams
coming from each device, which (as you mentioned) are all different,
because they all support different (although overlapping) extensions.

All of these calls that currently get dumped into ioctl (because they are
not important enough to warrant an API function at the top level) suddenly
become well structure and well designed interfaces.  For example, if your
device doesn't support the terminal handling ioctls the system has to
basically figure that out and report errors on it.  With CORBA/kORBit, you
would just not implement that interface... so if someone asked for it,
they would get a null pointer to the interface, which they check for and
realize that it's impossible to do terminal stuff on.

isatty (an example of an IOCTL wrapper) suddenly becomes implicit and the
API is cleaner...

I have no problem with the "unix way", but I do think that it can be
augmented in some ways...

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/










-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


> > There is a large perception of CORBA being slow, but for the most part it
> > is unjustified.  I believe that the act of _designing_ a completely new

> CORBA is slow compared to some of the other solutions. The question I was 
> trying to ask is whether you should put something smaller and faster into the
> kernel space and leave CORBA in userland. 

What I'm trying to show is that CORBA itself is not slow.  CORBA can be
thought of as an IDEA, and the current implementations are
suboptimal.  For example, no CORBA implementation has been tweaked to run
well in-kernel.  Because of this, people will say that CORBA sucks and is
slow, but that's not true.

I will agree that IIOP (the standardized corba communication mechanism) is
much slower and more complex than we would want for a generalized
user->kernel communication mechanism.  The nice thing about CORBA is that
you can design your own transport to optimize things.  Thus 99% of the
time you could use a heavily optimized, very simple transport that doesn't
do much.  This is what you are asking for, and it make perfect
sense.  What CORBA buys you is the ability to use this streamlined
interface without giving up full generality...

> It's complex, it has security 
> implications surely it belongs talking something simple and fast to the kernel.

First off, there are a lot of complex systems in the kernel.  :)
Second, there are not security implications above and beyond any normal
kernel hacking.  The reason that kORBit has security implications right
now is that there _IS NO SECURITY_.  Imagine implementing sys_read with
no checks of what uid is running, and you get the idea.  

Actually security in CORBA can be done BETTER in kernel space than in user
space, but I digress...

> If you look at microkernels they talk a much simpler faster rpc protocol. 

Yes but for the most part they lose all of the advantages of CORBA as
well.  For example, every microkernel that I'm aware of can only talk to
remote servers/clients that are running the same microkernel on the remote
machine.  There are no provisions for byte swapping if the endianness is
incorrect or for having structured bytestreams (so that you can tell
things about the raw bytes coming over the line without understanding them
all).

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


> > Of course.  Which is why CORBA is about putting STRUCTURE in that stream
> > of random bytes coming over the wire.  Why should I have to rewrite my
> > marshalling and demarshalling code every time I want to write a
> > server.  read and write are fine.  But sometimes I want a
> > structure.  Sometimes, my structures aren't laid out like C struct's
> > either.  What then?  What if I want to send an "object" to you?
> 
> Then I need to understand the object anyway. And Corba objects are horribly
> over complex. Any lisp hacker will tell you there is only one type: a list.

But alan, that's the beautiful thing.  Given a CORBA object, you can
understand its structure without knowing exactly what the contents
are.  You can effectively derive it's prototype just by inspecting it.

The only difference between lisp and a CORBA object in this respect is
that each item in the list is typed so that you have even more info about
what to do with it.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


> > > There is a large perception of CORBA being slow, but for the most part it
> > > is unjustified.  
> 
> Really?  I have that same perception but I can't claim that I've measured it.
> On the other hand, I have measured the overhead of straight UDP, TCP, and
> Sun RPC ping/pong tests and you can find the code for that in any version
> of lmbench.  It should be a 5 minute task for someone who groks corba to
> do the same thing using the same framework.  If someone wants to do it,
> I'll guide them through the lmbench stuff.  It's pretty trivial, start 

Urm... thanks for the offer... but you misunderstand me if you think that
I'm claiming that kORBit is the ideal/fast implementation that everyone
has been looking for.  There is still much to be done.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-14 Thread Chris Lattner


> > > Oh, great. So we don't have to care about formatting changes. We just
> > > have to care about the data changes. IOW, we are shielded from the
> > > results of changes that should never happen in the first place. And the
> > > benefit being...?
> > 
> > What the hell are you talking about?  Did you even read my example? I was
> > giving an example of extending an API, adding new functionality to it.
> 
> Yes, I did. What I don't understand is how kernel mechanism for marshalling
> would make your life easier wrt changes.

I gave a very simple example of how an interface could be designed and
then later extended without breaking any user space programs.  Please
reread the example and tell me precisely what doesn't make sense, and I
would be glad to explain it better.

> > > Notice also that I can say ls /dev/*mouse* and get some idea of the files
> > > there. I can't do that for your interfaces.

> > And you know what?  I can't do ls /dev/*net* and get all the network
> > devices either.  Actually, one of the very cool things about CORBA is that
> > you can BROWSE/LIST/SEARCH all objects currently instantiated.  Being
> > able to browse all in kernel objects would be very cool.

> So what's to stop you from letting readdir() do the work? WTF do we need
> one more API for returning the list of objects?

Urm.. that was the point.  You don't add a new API (it's built into corba,
along with introspection).  Also, like I JUST MENTIONED, readdir will have
no way of knowing what is a network device in the quagmire of stuff
floating around in /dev.  (ls *net* does actually call readdir ya
know.  Perhaps you should read up more on unix... [sorry couldn't help
it. ;])

> > > The point being: if your program spends efforts on marshalling it would
> > > better _do_ something with the obtained data. And then we are back to
> > > the square 1.
> > Uh huh, go ahead and read the example I sent to you.
> Did that.

So they you understand that no interface was broken even though more data
is now available...

> > > Returning to your example, I could not tee(1) the stream into file for
> > > later analysis. Not unless I write a special-case program for intercepting
> > > that stuff. I don't see why it is a good thing.

> > On the contrary, it would be pretty easy to do something like that with
> > CORBA.  No you wouldn't be able to use tee, but why would you want to tee
> > a binary data file?  The only reason that tee works in this situation is

> For the same reasons why I use tar, gzip, whatever. I don't _want_ to
> invent a new utility every time when Joe Doe adds a new piece of interface.
> Data is data is data. I can uuencode it and send to somebody who would
> care to analize the bloody thing. Do you mean that I need to write a
> special tool for that? For _every_ member of every interface somebody
> decided to add? I don't think so. I really don't.

That's fine, I don't blame you.  Ya know what?  I even agree with you
(image that?  hehe :).  You seem to forget that a CORBA object is exactly
the same thing: a stream of bytes.  Actually it does better than
that.  It's a stream of bytes that you can introspect on and determine the
structure of.  Imaging this.  You have a bytestream flowing out of some
pipe/device/socket/file/whatever and you pipe it to a
"Decode" program.  The decode program spits this out:

sequence < 
   struct { 
 float
 float
   }>

This example is meant to illustrate that _in a general way_ you can decode
the structure of the bytestream.  (hence my claims about CORBA adding
structure to a standard bytestream)...  this is quite a bit more useful
than trying to figure out what 'cat' is trying to tell you about binary
data (especially if it's floating point numbers).

> > line... your analysis program would have to have special purpose code to
> > parse the file.  EVERY consumer of "mouse data" would have to parse the

> Or I would look at the size. Or I would say od and look at the result. Don't
> tell me what to do with the data, when I'll need to parse it I will. And
> if you expect me to bother with writing more stuff when generic tools would
> work fine - too bad, I've been there, done that and I'm not coming back.

Heh... I'm not trying to replace od or cat or tee or pipes.  What I'm
trying to do is give you a larger toolkit that gives you MORE
power.  Imagine you want to get the first float value (of each struct) of
the above stream.  Imagine a functional scripting language where you could
do this (haskell syntax stolen gratitously):

cat /dev/floatymouse | funcwith 'map fst'

Which you could then point to the "decode" type program mentioned above,
and you would now get:

sequence

> BTW, you cared about size of /dev, didn't you? /usr/bin choke-full of tee-foo,
> tee-bar, yodda, yodda would be better?

You wouldn't have to replace the standard utilities.  Sorry to rain on
your parade.

> > file.  That seems pretty silly to me.
> Difference between good program and bad one: the 

Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-15 Thread Chris Lattner

> > For one of our demos, we ran a file server on a remote linux box (that we 
> > just had a user account on), mounted it on a kORBit'ized box, and ran
> > programs on SPARC Solaris that accessed the kORBit'ized linux box's file
> > syscalls.  If nothing else, it's pretty nifty what you can do in little
> > code...
> 
> Cool!
> 
> However, can you do one test for me? Do _heavy_ writes on kORBit-ized
> box. That might show you some problems. Oh, and try to eat atomic
> memory by ping -f kORBit-ized box.

I'll give that a try when I get a chance.  :)

> I've always wanted to do this: redirect /dev/dsp from one machine to
> another. (Like, I have development machine and old 386. I want all
> programs on devel machine use soundcard from 386. Can you do that?)

Yes.  Definately.  There are probably other ways of doing that... but one
of the things we implemented was a "generic" character device... and we
tested it by having a chardev server that basically reads from a
"local" (to the server) character device, and forward it over CORBA.  So
this is already implemented!  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit

2000-12-18 Thread Chris Lattner

> >   cat /mnt/www/www.kernel.org/index.html
> 
> can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came
> to conclusion that web filesystem is not possible... (If you can't do

Yes, if the server supports webDAV or something similar.

> listings, it is not really filesystem; you could do
> 
>  cat /mnt/www/www.kernel.org_index.html as well, and that's easy to
> do.) 
> 
> > and the CorbaFS userspace server takes care of loading the webpage and
> > returning it to the kernel client.  And these new filesystems don't
> > take up any extra space in the kernel, since they all talk to the same
> > CorbaFS kernel module!  Not to mention being able to implement the
> > filesystem in any language you like, debug the implementation in
> > userspace, etc.
> 
> codafs can do pretty much the same.

Yes, but codaFS is specific to filesystems.  kORBit, of course, can do
much much more, in a very uniform way.  :)

-Chris

http://www.nondot.org/~sabre/os/
http://www.nondot.org/MagicStats/
http://korbit.sourceforge.net/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/



Re: [RFC] bloody mess with __attribute__() syntax

2007-07-05 Thread Chris Lattner

On Thu, 5 Jul 2007, Al Viro wrote:

On Thu, Jul 05, 2007 at 09:41:55AM -0700, Linus Torvalds wrote:

Note that gcc rules for __attribute__() (and that's the only source
of rules we _have_ for the damn thing) clearly say that
int __user *p;
is the same thing as
int *__user p;


Quick question: is there some reason why we have to honor the crazy gcc
rules, and cannot try to convince gcc people that they are insane?


AFAICS, they started with storage-class-like attributes.  Consider e.g.
always_inline or section; these are not qualifiers at all and you want
to have
static __attribute__((always_inline)) int foo(int *p);
interpreted with attribute applied to foo, not to its return type.


This is true, but I don't think this is related.  attributes in GCC can 
apply either to types or two decls.  In this case, the always_inline 
attribute is being applied to the decl, but other attributes could be 
applied to the return type.


-Chris

--
http://nondot.org/sabre/
http://llvm.org/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/