Dave,
Here's some feedback. I don't have access to source code for any branded
Unix. (I'm still in the clean room, which is good.) From the Solaris and
Unixware documentation, however, it is clear that when the "clone" driver
is used, that a "new and unique minor number" must be assigned by the
driver. I have also seen statements (although I can't remember just where)
that the driver must not assign a new major device number (it appears that
only "clone" is allowed to do this). In Magic Garden it explains that the
SVR4 STREAMS executive performs a setq() after qi_open of the driver if the
major device number changes, but this is for "clone" I believe. LiS "clone"
peforms the setq() itself within its qi_open routine.
I believe the original purpose in LiS of redirecting to an open (or existing)
stream head via the device number was for the "connld" module, which opens
a pipe and then uses the pipe's character device number (major and minor) to
redirect the LiS executive to use the new pipe end. In SVR4 pipes use a
different file system from normal streams devices, even if they are
STREAMS-based bi-directional pipes.
Unixware documentation, on the other hand talks of "clonable minors" and
"clone channels".
In practice if the minor device number is really meant to be clonable, then
there is no reason to assign a new or unique device number. The following is
pretty much the situation:
When opened via "clone" the driver must assign a new and unique device number.
If it does not, yet does not return an error number to qi_open() then the
executive should close the device by calling qi_close() and return ENXIO to
the open(2) call, particularly if filesystem corruption would result down
the road.
When a minor device is marked as cloneable (SAP_CLONE directive to the uw7
SAD driver, or implementation specific d_flags on initialization), then the
driver should be permitted to reuse the same minor device number. This has
one ramification for clonable minors: only one stream head can be accessed
by an explicit open of the minor device. Clonable minors can get around this
my marking the minor clonable and never permitting an explicit open of the
minor device (a new stream head is always created when the minor is opened).
The special file system must of course permit this.
For "clone," because minor devices can subsequently be opened explicitly
without a clone open (DRVOPEN), there should be one stream head per special
file. It is possible, however, to even accomodate this by stacking stream
heads behind the special file and only permitting explicit open of the
topmost stream-head in the stack.
This latter is the approach that I took with Linux Fast-STREAMS. If a
"clone" open returns a device number that already exists, the new stream
head is used and the stream head is "stacked" on the special file. A clone
open (whether via "clone" or a clonable minor) will always result in a new
stream head "stacked" in this fashion. In the case of "clone" there should
be only one stream head in the stack if the documentation rules ("new and
unique minor device number assigned") are followed. An explicit open of the
device number (DRVOPEN) is performed, then the topmost stream head in the
"stack" has its qi_open() procedure called.
This suits my pusposes well and seems consistent with the uw7 documentation.
I have the difficulty that I need to open thousands of streams for
configuration of SS7 protocol stacks. Under LiS, I used to dynamically assign
new major device numbers (sometimes on demand), and used the capability that
LiS allows you to reassign the major device number on open. (In fact LiS
allows you to reassign the major and minor device number on a DRVOPEN, which
made things easier). This is consistent with the "new and unique"
requirement. The strinet driver does this too. Although it has clone minors
(/dev/tcp, /dev/udp etc. are all minor device numbers under the socksys major)
a new unique major/minor device number combination is assigned for each
stream.
One problem with this approach was that I had applications where I needed to
open 10's of thousands of streams, linked under multiplexing drivers and the
like. Almost all these streams are pseudo-devices and few need to be opened
explicitly via major/minor device number. The system could quite quickly run
out of device numbers altogether it the "new and unique" approach is used.
(Linux at one time used to associate a character device with each socket
opened. Failure to scale is why it was dropped.) The clonable minor approach
allows any number of stream heads to be stacked behind the same minor device
number, yet is compatible with the old "clone" method.
It look's like "clone" goes all the way back to Ritchie's Research UNIX,
where the original "new and unique" requirement came from.
It is in fact a lot cleaner that if a clone opened driver returns an existing
major/minor device number that the stream head be "stacked" on that special
file rather than anything else. This is because if the driver has returned
zero (0) it has attached (or could have attached) private structures to the
passed in queue pair. This stream head should not be freed without calling
the driver's qi_close() procedure so that it can remove its private
structures from the queue pair. So I suppose the choice is to "stack" the
stream or to call qi_close() and return ENXIO to open(2). I think the only
reason LiS is doing any different right now is because "connld" uses this
corner to redirect the open to the new existing pipe end. I rewrote "connld"
for Linux Fast-STREAMS so that this behavior was no longer necessary ("connld"
more directly installs the stream head for the new pipe end against the file
descriptor.) This was a lot easier (it seems) in SVR 4 because SVR 4 returns
the vnode as a return to c_open calling spec_open. Linux must actually
install dentry and vfsmount against the file pointer.
One more note about stacking stream heads: the file->private_data pointer
always points to the stream head in the stack that belongs to an open file
descriptor. Thus a file pointer only references one stream head during its
lifetime. An inode, on the other hand can reference multiple stream heads.
I wish I could point you to the code for Linux Fast-STREAMS so you could
follow what I have done as well, but I'm still packaging for release...
--brian
On Mon, 06 Oct 2003, Dave Grothe wrote:
> I have the old multi-threaded open/close race problem fixed, or at least I
> will claim that it is fixed until someone produces an example that
> demonstrates otherwise.
>
> Before releasing the LiS incorporating these changes I need to solicit your
> opinions on a matter.
>
> When a user process opens a specific stream {maj,min} for a second or
> subsequent open, LiS (and Solaris) calls the driver open routines for all
> the modules in the stream to announce this fact. These open calls are not
> balanced by calls to the driver close routine, since the close routine is
> called only on the LAST close on the stream.
>
> So far, so good.
>
> When a clone open occurs the initial open is to {clone,maj} with the
> driver's open routine being called with {maj,0} and the clone flag
> set. The driver returns an updated {maj,min} to use. If this {maj,min} is
> currently not open then STREAMS simply uses the stream head structure that
> was originally for {clone,maj} to operate {maj,min}.
>
> So far, so good.
>
> But what happens if a clone open returns a {maj,min} that is already
> open? At the moment, LiS does NOT call the open routine of the driver that
> owns {maj,min}, on the theory that the driver open routine was already
> called via the clone driver.
>
> But is that correct? An alternate theory is that a clone open to an
> already-open {maj,min} could be seen as a indirect open to that stream and
> that normal re-open semantics would be to call the open routines.
>
> Does anyone have any direct evidence of what other STREAMS implementations
> do in this case? I don't have a Solaris test case for this.
>
> Thanks,
> Dave
>
>
> _______________________________________________
> Linux-streams mailing list
> [EMAIL PROTECTED]
> http://gsyc.escet.urjc.es/mailman/listinfo/linux-streams
--
Brian F. G. Bidulock � The reasonable man adapts himself to the �
[EMAIL PROTECTED] � world; the unreasonable one persists in �
http://www.openss7.org/ � trying to adapt the world to himself. �
� Therefore all progress depends on the �
� unreasonable man. -- George Bernard Shaw �
_______________________________________________
Linux-streams mailing list
[EMAIL PROTECTED]
http://gsyc.escet.urjc.es/mailman/listinfo/linux-streams
