Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
[EMAIL PROTECTED] writes: > | No. The "other" device namespace I would construct on machine B to > | look just like the device namespace that existed on machine A. > | Making /sys/devices/block/sda would still be 8:0. > | > | So to be very clear on machine B when talking about disk-1 I would have. > | initial device namespace: > | /sys/devices/block/sdb > | /sys/devices/block/sdb/dev 8:16 > | > | "other" device namespace: > | /sys/devices/block/sda > | /sys/devices/block/sda/dev 8:0 > | > | Similarly on machine B when talking about disk-2 I would have. > | initial device namespace: > | /sys/devices/block/sda > | /sys/devices/block/sda/dev 8:0 > | > | "other" device namespace: > | /sys/devices/block/sdb > | /sys/devices/block/sdb/dev 8:16 > | > | So between the two devices namespaces on machine B the two disks > | would exchange their user visible identities. > > So an application that would migrate from machine A to B has to > use virtual names (like "disk-1" and "disk-2") to access the disk > right ? No. It is worse you need to access a filesystem and probably a block device that is available on both machine A and machine B. With care we can introduce appropriate namespaces and namespace semantics so we can make the names be what we need. For a classic tricky case think what it would require to migrate a git archive with checked out files and not need to say "git-update-index --refresh" before you work with the files. I used names like disk-1 and disk-2 instead of UUIDs because it was easier for me to type and think about. You do need some kind of absolute disk or filesystem identity you can refer back to. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Eric W. Biederman [EMAIL PROTECTED] wrote: | Greg KH <[EMAIL PROTECTED]> writes: | | > On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: | >> Greg KH <[EMAIL PROTECTED]> writes: | >> > | >> >> Also fun is that the dev file implementation needs to be able to | >> >> report different major:minor numbers based on which mount of | >> >> sysfs we are dealing with. | >> > | >> > Um, no, that's not going to happen. /dev/sda will _always_ have the | >> > same major:minor number, as defined by the LSB spec. You can not break | >> > that at all. So while you might not want to show all mounts | >> > /sys/devices/block/sda/ the ones that you do, will all have the LSB | >> > defined major:minor number assigned to it. | >> | >> Hmm. If that is in the LSB it must come from | >> Documentation/devices.txt | > | > Yes, that is the requirement. | > | >> I'm not after changing the user visible major/minor assignments. | > | > Oh, I misunderstood what you wrote above then. | | My above sentence is slightly misleading. That should have been. | I am not after changing the device name to major:minor assignments | as specified in Documentation/devices.txt. | | So within a single device namespace everything is normal and as it | always has been. Weirdness only ensues when you look across device | namespaces. | | >> Let me see if a concrete example will help. Suppose I have | >> have a SAN with two disks: disk-1 and disk-2. I have | >> two machines A and B. On machine A I get the mapping: | >> sda -> disk-1, sdb ->disk-2. On machine B I wind up with | >> a different probe order so I get the mapping: sda -> disk-2 | >> sdb ->disk-1. | > | > Ok. | > | >> To be very clear by sda I mean the block device with major 8 and | >> minor 0, and by sdb I mean the block device with major 8 and minor | >> 16. | > | > Ok. | > | >> So I decide I want an environment on machine B that looks just | >> like the environment on machine A, so I can bring transfer over | >> a running program or whatever. So I run around looking at UUID | >> labels and what not and I discover that the machine B knows disk-1 as | >> sdb and that machine A knows disk-1 as sda. So I want to say: | >> /sys/devices/block/sdb show up in this other device namespace as | >> /sys/devices/block/sda. | | > | > Ah, but if you do that then the "other" device namespace would have | > /sys/devices/block/sda/dev be 8:16, right? | | No. The "other" device namespace I would construct on machine B to | look just like the device namespace that existed on machine A. | Making /sys/devices/block/sda would still be 8:0. | | So to be very clear on machine B when talking about disk-1 I would have. | initial device namespace: | /sys/devices/block/sdb | /sys/devices/block/sdb/dev 8:16 | | "other" device namespace: | /sys/devices/block/sda | /sys/devices/block/sda/dev 8:0 | | Similarly on machine B when talking about disk-2 I would have. | initial device namespace: | /sys/devices/block/sda | /sys/devices/block/sda/dev 8:0 | | "other" device namespace: | /sys/devices/block/sdb | /sys/devices/block/sdb/dev 8:16 | | So between the two devices namespaces on machine B the two disks | would exchange their user visible identities. So an application that would migrate from machine A to B has to use virtual names (like "disk-1" and "disk-2") to access the disk right ? | | Eric | ___ | Containers mailing list | [EMAIL PROTECTED] | https://lists.linux-foundation.org/mailman/listinfo/containers - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Eric W. Biederman [EMAIL PROTECTED] wrote: | Greg KH [EMAIL PROTECTED] writes: | | On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: | Greg KH [EMAIL PROTECTED] writes: | | Also fun is that the dev file implementation needs to be able to | report different major:minor numbers based on which mount of | sysfs we are dealing with. | | Um, no, that's not going to happen. /dev/sda will _always_ have the | same major:minor number, as defined by the LSB spec. You can not break | that at all. So while you might not want to show all mounts | /sys/devices/block/sda/ the ones that you do, will all have the LSB | defined major:minor number assigned to it. | | Hmm. If that is in the LSB it must come from | Documentation/devices.txt | | Yes, that is the requirement. | | I'm not after changing the user visible major/minor assignments. | | Oh, I misunderstood what you wrote above then. | | My above sentence is slightly misleading. That should have been. | I am not after changing the device name to major:minor assignments | as specified in Documentation/devices.txt. | | So within a single device namespace everything is normal and as it | always has been. Weirdness only ensues when you look across device | namespaces. | | Let me see if a concrete example will help. Suppose I have | have a SAN with two disks: disk-1 and disk-2. I have | two machines A and B. On machine A I get the mapping: | sda - disk-1, sdb -disk-2. On machine B I wind up with | a different probe order so I get the mapping: sda - disk-2 | sdb -disk-1. | | Ok. | | To be very clear by sda I mean the block device with major 8 and | minor 0, and by sdb I mean the block device with major 8 and minor | 16. | | Ok. | | So I decide I want an environment on machine B that looks just | like the environment on machine A, so I can bring transfer over | a running program or whatever. So I run around looking at UUID | labels and what not and I discover that the machine B knows disk-1 as | sdb and that machine A knows disk-1 as sda. So I want to say: | /sys/devices/block/sdb show up in this other device namespace as | /sys/devices/block/sda. | | | Ah, but if you do that then the other device namespace would have | /sys/devices/block/sda/dev be 8:16, right? | | No. The other device namespace I would construct on machine B to | look just like the device namespace that existed on machine A. | Making /sys/devices/block/sda would still be 8:0. | | So to be very clear on machine B when talking about disk-1 I would have. | initial device namespace: | /sys/devices/block/sdb | /sys/devices/block/sdb/dev 8:16 | | other device namespace: | /sys/devices/block/sda | /sys/devices/block/sda/dev 8:0 | | Similarly on machine B when talking about disk-2 I would have. | initial device namespace: | /sys/devices/block/sda | /sys/devices/block/sda/dev 8:0 | | other device namespace: | /sys/devices/block/sdb | /sys/devices/block/sdb/dev 8:16 | | So between the two devices namespaces on machine B the two disks | would exchange their user visible identities. So an application that would migrate from machine A to B has to use virtual names (like disk-1 and disk-2) to access the disk right ? | | Eric | ___ | Containers mailing list | [EMAIL PROTECTED] | https://lists.linux-foundation.org/mailman/listinfo/containers - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
[EMAIL PROTECTED] writes: | No. The other device namespace I would construct on machine B to | look just like the device namespace that existed on machine A. | Making /sys/devices/block/sda would still be 8:0. | | So to be very clear on machine B when talking about disk-1 I would have. | initial device namespace: | /sys/devices/block/sdb | /sys/devices/block/sdb/dev 8:16 | | other device namespace: | /sys/devices/block/sda | /sys/devices/block/sda/dev 8:0 | | Similarly on machine B when talking about disk-2 I would have. | initial device namespace: | /sys/devices/block/sda | /sys/devices/block/sda/dev 8:0 | | other device namespace: | /sys/devices/block/sdb | /sys/devices/block/sdb/dev 8:16 | | So between the two devices namespaces on machine B the two disks | would exchange their user visible identities. So an application that would migrate from machine A to B has to use virtual names (like disk-1 and disk-2) to access the disk right ? No. It is worse you need to access a filesystem and probably a block device that is available on both machine A and machine B. With care we can introduce appropriate namespaces and namespace semantics so we can make the names be what we need. For a classic tricky case think what it would require to migrate a git archive with checked out files and not need to say git-update-index --refresh before you work with the files. I used names like disk-1 and disk-2 instead of UUIDs because it was easier for me to type and think about. You do need some kind of absolute disk or filesystem identity you can refer back to. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH <[EMAIL PROTECTED]> writes: > Ah, ok, that makes more sense. > > And seems quite difficult to do, good luck with that :) Thanks. At least now all I have to do is worry about the details when we get that far instead of selling the big picture... My gut feel is that sysfs is probably the hardest part to deal with, and maybe half of the problem. Just intercepting the lookup by device number is fairly simple, I think there is one spot for block devices and another for character devices. Anyway once the network namespace support is in with any luck that will have solved half the sysfs problem. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, Oct 10, 2007 at 07:16:48AM -0600, Eric W. Biederman wrote: > Greg KH <[EMAIL PROTECTED]> writes: > > > On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: > >> Greg KH <[EMAIL PROTECTED]> writes: > >> > > >> >> Also fun is that the dev file implementation needs to be able to > >> >> report different major:minor numbers based on which mount of > >> >> sysfs we are dealing with. > >> > > >> > Um, no, that's not going to happen. /dev/sda will _always_ have the > >> > same major:minor number, as defined by the LSB spec. You can not break > >> > that at all. So while you might not want to show all mounts > >> > /sys/devices/block/sda/ the ones that you do, will all have the LSB > >> > defined major:minor number assigned to it. > >> > >> Hmm. If that is in the LSB it must come from > >> Documentation/devices.txt > > > > Yes, that is the requirement. > > > >> I'm not after changing the user visible major/minor assignments. > > > > Oh, I misunderstood what you wrote above then. > > My above sentence is slightly misleading. That should have been. > I am not after changing the device name to major:minor assignments > as specified in Documentation/devices.txt. > > So within a single device namespace everything is normal and as it > always has been. Weirdness only ensues when you look across device > namespaces. > > >> Let me see if a concrete example will help. Suppose I have > >> have a SAN with two disks: disk-1 and disk-2. I have > >> two machines A and B. On machine A I get the mapping: > >> sda -> disk-1, sdb ->disk-2. On machine B I wind up with > >> a different probe order so I get the mapping: sda -> disk-2 > >> sdb ->disk-1. > > > > Ok. > > > >> To be very clear by sda I mean the block device with major 8 and > >> minor 0, and by sdb I mean the block device with major 8 and minor > >> 16. > > > > Ok. > > > >> So I decide I want an environment on machine B that looks just > >> like the environment on machine A, so I can bring transfer over > >> a running program or whatever. So I run around looking at UUID > >> labels and what not and I discover that the machine B knows disk-1 as > >> sdb and that machine A knows disk-1 as sda. So I want to say: > >> /sys/devices/block/sdb show up in this other device namespace as > >> /sys/devices/block/sda. > > > > > Ah, but if you do that then the "other" device namespace would have > > /sys/devices/block/sda/dev be 8:16, right? > > No. The "other" device namespace I would construct on machine B to > look just like the device namespace that existed on machine A. > Making /sys/devices/block/sda would still be 8:0. > > So to be very clear on machine B when talking about disk-1 I would have. > initial device namespace: > /sys/devices/block/sdb > /sys/devices/block/sdb/dev 8:16 > > "other" device namespace: > /sys/devices/block/sda > /sys/devices/block/sda/dev 8:0 > > Similarly on machine B when talking about disk-2 I would have. > initial device namespace: > /sys/devices/block/sda > /sys/devices/block/sda/dev 8:0 > > "other" device namespace: > /sys/devices/block/sdb > /sys/devices/block/sdb/dev 8:16 > > So between the two devices namespaces on machine B the two disks > would exchange their user visible identities. Ah, ok, that makes more sense. And seems quite difficult to do, good luck with that :) greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, Oct 10, 2007 at 11:26:32AM -0700, Martin Bligh wrote: > Greg KH wrote: >> On Wed, Oct 10, 2007 at 10:24:43AM -0700, Martin Bligh wrote: The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for "pass-through" type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) - when a directory is created/removed, a uevent should happen declaring what type of device was created/removed. >>> So you'll be removing: >>> >>> /sys/devices/system/node/node?/meminfo >>> >>> then? >>> >>> along with: >>> >>> /sys/devices/system/node/node?/distance >>> /sys/devices/system/node/node?/numastat >>> >>> and all the other things that violate the rules? >> I would love to do that :) >> And that goes to show how trying to enforce these kinds of rules is damm >> hard. Things slip by that I never notice because they are only for odd >> types of hardware :) > > Is there no way to enforce that in the sysfs interface? > (Haven't looked, I'll admit). Hm, we could parse the buffer from the response and complain if we notice spaces in it :) >>> (which I do agree with ... I just don't think sysfs works for >>> performance stats as we've discussed several times before ;-)) >> I agree that this doesn't work too, but also that if it's really needed, >> it can be done, just let us know about it (like >> /sys/block/BLOCKDEV/stat) > > OK. Would be nice if we could get rid of /sys/devices/system at > some point, which seems like a fairly crazy path, but still ;-) I agree, getting rid of the sysdev stuff is on the list of things I want to see changed, that code is not nice... thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH wrote: On Wed, Oct 10, 2007 at 10:24:43AM -0700, Martin Bligh wrote: The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for "pass-through" type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) - when a directory is created/removed, a uevent should happen declaring what type of device was created/removed. So you'll be removing: /sys/devices/system/node/node?/meminfo then? along with: /sys/devices/system/node/node?/distance /sys/devices/system/node/node?/numastat and all the other things that violate the rules? I would love to do that :) And that goes to show how trying to enforce these kinds of rules is damm hard. Things slip by that I never notice because they are only for odd types of hardware :) Is there no way to enforce that in the sysfs interface? (Haven't looked, I'll admit). (which I do agree with ... I just don't think sysfs works for performance stats as we've discussed several times before ;-)) I agree that this doesn't work too, but also that if it's really needed, it can be done, just let us know about it (like /sys/block/BLOCKDEV/stat) OK. Would be nice if we could get rid of /sys/devices/system at some point, which seems like a fairly crazy path, but still ;-) M. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, Oct 10, 2007 at 10:24:43AM -0700, Martin Bligh wrote: >> The rules for sysfs files are the following: >> - one value, in text format, per file. >> - no action apon open/close >> - binary files are only allowed for "pass-through" type files >>that the kernel does not touch (like for firmware and pci >>config space) >> - directories should be associated with a kobject where it makes >>sense (no nesting deep subdirectories without a kobject >>present) >> - when a directory is created/removed, a uevent should happen >>declaring what type of device was created/removed. > > So you'll be removing: > > /sys/devices/system/node/node?/meminfo > > then? > > along with: > > /sys/devices/system/node/node?/distance > /sys/devices/system/node/node?/numastat > > and all the other things that violate the rules? I would love to do that :) And that goes to show how trying to enforce these kinds of rules is damm hard. Things slip by that I never notice because they are only for odd types of hardware :) > (which I do agree with ... I just don't think sysfs works for > performance stats as we've discussed several times before ;-)) I agree that this doesn't work too, but also that if it's really needed, it can be done, just let us know about it (like /sys/block/BLOCKDEV/stat) thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for "pass-through" type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) - when a directory is created/removed, a uevent should happen declaring what type of device was created/removed. So you'll be removing: /sys/devices/system/node/node?/meminfo then? along with: /sys/devices/system/node/node?/distance /sys/devices/system/node/node?/numastat and all the other things that violate the rules? (which I do agree with ... I just don't think sysfs works for performance stats as we've discussed several times before ;-)) M. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, 10 Oct 2007 11:38:50 -0400 (EDT), Alan Stern <[EMAIL PROTECTED]> wrote: > On Tue, 9 Oct 2007, Greg KH wrote: > > > The rules for sysfs files are the following: > > - one value, in text format, per file. > > - no action apon open/close > > - binary files are only allowed for "pass-through" type files > > that the kernel does not touch (like for firmware and pci > > config space) > > - directories should be associated with a kobject where it makes > > sense (no nesting deep subdirectories without a kobject > > present) > > You have to stretch this a little for the power/ subdirectory every > device gets. The only kobject it corresponds to is the one in the > device, which already corresponds to the parent directory. Yes, this should be "directories should be associated with a kobject or generated by an attribute group". This way you won't get deep nesting either. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, 9 Oct 2007, Greg KH wrote: > The rules for sysfs files are the following: > - one value, in text format, per file. > - no action apon open/close > - binary files are only allowed for "pass-through" type files > that the kernel does not touch (like for firmware and pci > config space) > - directories should be associated with a kobject where it makes > sense (no nesting deep subdirectories without a kobject > present) You have to stretch this a little for the power/ subdirectory every device gets. The only kobject it corresponds to is the one in the device, which already corresponds to the parent directory. Alan Stern - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH <[EMAIL PROTECTED]> writes: > On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: >> Greg KH <[EMAIL PROTECTED]> writes: >> > >> >> Also fun is that the dev file implementation needs to be able to >> >> report different major:minor numbers based on which mount of >> >> sysfs we are dealing with. >> > >> > Um, no, that's not going to happen. /dev/sda will _always_ have the >> > same major:minor number, as defined by the LSB spec. You can not break >> > that at all. So while you might not want to show all mounts >> > /sys/devices/block/sda/ the ones that you do, will all have the LSB >> > defined major:minor number assigned to it. >> >> Hmm. If that is in the LSB it must come from >> Documentation/devices.txt > > Yes, that is the requirement. > >> I'm not after changing the user visible major/minor assignments. > > Oh, I misunderstood what you wrote above then. My above sentence is slightly misleading. That should have been. I am not after changing the device name to major:minor assignments as specified in Documentation/devices.txt. So within a single device namespace everything is normal and as it always has been. Weirdness only ensues when you look across device namespaces. >> Let me see if a concrete example will help. Suppose I have >> have a SAN with two disks: disk-1 and disk-2. I have >> two machines A and B. On machine A I get the mapping: >> sda -> disk-1, sdb ->disk-2. On machine B I wind up with >> a different probe order so I get the mapping: sda -> disk-2 >> sdb ->disk-1. > > Ok. > >> To be very clear by sda I mean the block device with major 8 and >> minor 0, and by sdb I mean the block device with major 8 and minor >> 16. > > Ok. > >> So I decide I want an environment on machine B that looks just >> like the environment on machine A, so I can bring transfer over >> a running program or whatever. So I run around looking at UUID >> labels and what not and I discover that the machine B knows disk-1 as >> sdb and that machine A knows disk-1 as sda. So I want to say: >> /sys/devices/block/sdb show up in this other device namespace as >> /sys/devices/block/sda. > > Ah, but if you do that then the "other" device namespace would have > /sys/devices/block/sda/dev be 8:16, right? No. The "other" device namespace I would construct on machine B to look just like the device namespace that existed on machine A. Making /sys/devices/block/sda would still be 8:0. So to be very clear on machine B when talking about disk-1 I would have. initial device namespace: /sys/devices/block/sdb /sys/devices/block/sdb/dev 8:16 "other" device namespace: /sys/devices/block/sda /sys/devices/block/sda/dev 8:0 Similarly on machine B when talking about disk-2 I would have. initial device namespace: /sys/devices/block/sda /sys/devices/block/sda/dev 8:0 "other" device namespace: /sys/devices/block/sdb /sys/devices/block/sdb/dev 8:16 So between the two devices namespaces on machine B the two disks would exchange their user visible identities. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, 9 Oct 2007 16:28:04 -0700, Greg KH <[EMAIL PROTECTED]> wrote: > On Tue, Oct 09, 2007 at 04:20:39PM -0700, Roland Dreier wrote: > > > > - relaying implementation details to userspace so that they cannot be > > > > easily changed. We would need to allow kobjects not showing up in sysfs > > > > and making symlinks a sysfs facility not relying on kobjects to help > > > > there. > > > > > > Huh? Why would you want a kobject to not show up in sysfs? > > > > I think the text you replied to explained it perfectly: becase you > > don't want internal implementation details to be exposed the userspace > > and become an unchangeable part of the kernel/userspace interface. > > But a kobject represents "something" in the kernel. If it's there, then > it shows up in sysfs. But if it isn't, or it changes somehow, then it > no longer is in sysfs, which is fine, and your userspace tools have to > be able to handle that by virtue of the rules of how to use sysfs from > userspace. The rules for using sysfs information are currently only laid out for the driver model. We would need similar rules for every other user of kobjects. This only works if we make the following the general rules: - You may not rely on any information in sysfs to be present. - Exceptions to that rule are documented for that special user of sysfs. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, 9 Oct 2007 15:26:38 -0700, Greg KH <[EMAIL PROTECTED]> wrote: > > - relaying implementation details to userspace so that they cannot be > > easily changed. We would need to allow kobjects not showing up in sysfs > > and making symlinks a sysfs facility not relying on kobjects to help > > there. > > Huh? Why would you want a kobject to not show up in sysfs? If we consider everything that shows up in sysfs as API, we cannot go ahead and change it without risking userspace breakage. The hierarchy of objects may very well be an "implementation detail". Otherwise, we would need to lay out for every user of the kobject infrastructure which information userspace can rely on (like it is documented for the driver model). > And yes, I agree we could use some more "automatic" help in regards to > symlinks in sysfs when we change kobjects around. That would make > things simpler for the kobject core. Yes, non-automatic symlink handling is a large source of pain and errors. They should probably be tied to sysfs_dirents instead of kobjects. > > > Removing kobject from the interface doesn't change anything about this. > > > > Hm. Currently you have a file<->attribute correlation. This would > > change if you allow non-attribute files. > > I don't want to have non-attribute files, that's my main point here. That's what I tried to say as well :) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, 9 Oct 2007 15:26:38 -0700, Greg KH [EMAIL PROTECTED] wrote: - relaying implementation details to userspace so that they cannot be easily changed. We would need to allow kobjects not showing up in sysfs and making symlinks a sysfs facility not relying on kobjects to help there. Huh? Why would you want a kobject to not show up in sysfs? If we consider everything that shows up in sysfs as API, we cannot go ahead and change it without risking userspace breakage. The hierarchy of objects may very well be an implementation detail. Otherwise, we would need to lay out for every user of the kobject infrastructure which information userspace can rely on (like it is documented for the driver model). And yes, I agree we could use some more automatic help in regards to symlinks in sysfs when we change kobjects around. That would make things simpler for the kobject core. Yes, non-automatic symlink handling is a large source of pain and errors. They should probably be tied to sysfs_dirents instead of kobjects. Removing kobject from the interface doesn't change anything about this. Hm. Currently you have a file-attribute correlation. This would change if you allow non-attribute files. I don't want to have non-attribute files, that's my main point here. That's what I tried to say as well :) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, 9 Oct 2007 16:28:04 -0700, Greg KH [EMAIL PROTECTED] wrote: On Tue, Oct 09, 2007 at 04:20:39PM -0700, Roland Dreier wrote: - relaying implementation details to userspace so that they cannot be easily changed. We would need to allow kobjects not showing up in sysfs and making symlinks a sysfs facility not relying on kobjects to help there. Huh? Why would you want a kobject to not show up in sysfs? I think the text you replied to explained it perfectly: becase you don't want internal implementation details to be exposed the userspace and become an unchangeable part of the kernel/userspace interface. But a kobject represents something in the kernel. If it's there, then it shows up in sysfs. But if it isn't, or it changes somehow, then it no longer is in sysfs, which is fine, and your userspace tools have to be able to handle that by virtue of the rules of how to use sysfs from userspace. The rules for using sysfs information are currently only laid out for the driver model. We would need similar rules for every other user of kobjects. This only works if we make the following the general rules: - You may not rely on any information in sysfs to be present. - Exceptions to that rule are documented for that special user of sysfs. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH [EMAIL PROTECTED] writes: On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: Greg KH [EMAIL PROTECTED] writes: Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Um, no, that's not going to happen. /dev/sda will _always_ have the same major:minor number, as defined by the LSB spec. You can not break that at all. So while you might not want to show all mounts /sys/devices/block/sda/ the ones that you do, will all have the LSB defined major:minor number assigned to it. Hmm. If that is in the LSB it must come from Documentation/devices.txt Yes, that is the requirement. I'm not after changing the user visible major/minor assignments. Oh, I misunderstood what you wrote above then. My above sentence is slightly misleading. That should have been. I am not after changing the device name to major:minor assignments as specified in Documentation/devices.txt. So within a single device namespace everything is normal and as it always has been. Weirdness only ensues when you look across device namespaces. Let me see if a concrete example will help. Suppose I have have a SAN with two disks: disk-1 and disk-2. I have two machines A and B. On machine A I get the mapping: sda - disk-1, sdb -disk-2. On machine B I wind up with a different probe order so I get the mapping: sda - disk-2 sdb -disk-1. Ok. To be very clear by sda I mean the block device with major 8 and minor 0, and by sdb I mean the block device with major 8 and minor 16. Ok. So I decide I want an environment on machine B that looks just like the environment on machine A, so I can bring transfer over a running program or whatever. So I run around looking at UUID labels and what not and I discover that the machine B knows disk-1 as sdb and that machine A knows disk-1 as sda. So I want to say: /sys/devices/block/sdb show up in this other device namespace as /sys/devices/block/sda. Ah, but if you do that then the other device namespace would have /sys/devices/block/sda/dev be 8:16, right? No. The other device namespace I would construct on machine B to look just like the device namespace that existed on machine A. Making /sys/devices/block/sda would still be 8:0. So to be very clear on machine B when talking about disk-1 I would have. initial device namespace: /sys/devices/block/sdb /sys/devices/block/sdb/dev 8:16 other device namespace: /sys/devices/block/sda /sys/devices/block/sda/dev 8:0 Similarly on machine B when talking about disk-2 I would have. initial device namespace: /sys/devices/block/sda /sys/devices/block/sda/dev 8:0 other device namespace: /sys/devices/block/sdb /sys/devices/block/sdb/dev 8:16 So between the two devices namespaces on machine B the two disks would exchange their user visible identities. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, 9 Oct 2007, Greg KH wrote: The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for pass-through type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) You have to stretch this a little for the power/ subdirectory every device gets. The only kobject it corresponds to is the one in the device, which already corresponds to the parent directory. Alan Stern - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, 10 Oct 2007 11:38:50 -0400 (EDT), Alan Stern [EMAIL PROTECTED] wrote: On Tue, 9 Oct 2007, Greg KH wrote: The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for pass-through type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) You have to stretch this a little for the power/ subdirectory every device gets. The only kobject it corresponds to is the one in the device, which already corresponds to the parent directory. Yes, this should be directories should be associated with a kobject or generated by an attribute group. This way you won't get deep nesting either. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for pass-through type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) - when a directory is created/removed, a uevent should happen declaring what type of device was created/removed. So you'll be removing: /sys/devices/system/node/node?/meminfo then? along with: /sys/devices/system/node/node?/distance /sys/devices/system/node/node?/numastat and all the other things that violate the rules? (which I do agree with ... I just don't think sysfs works for performance stats as we've discussed several times before ;-)) M. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, Oct 10, 2007 at 10:24:43AM -0700, Martin Bligh wrote: The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for pass-through type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) - when a directory is created/removed, a uevent should happen declaring what type of device was created/removed. So you'll be removing: /sys/devices/system/node/node?/meminfo then? along with: /sys/devices/system/node/node?/distance /sys/devices/system/node/node?/numastat and all the other things that violate the rules? I would love to do that :) And that goes to show how trying to enforce these kinds of rules is damm hard. Things slip by that I never notice because they are only for odd types of hardware :) (which I do agree with ... I just don't think sysfs works for performance stats as we've discussed several times before ;-)) I agree that this doesn't work too, but also that if it's really needed, it can be done, just let us know about it (like /sys/block/BLOCKDEV/stat) thanks, greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH wrote: On Wed, Oct 10, 2007 at 10:24:43AM -0700, Martin Bligh wrote: The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for pass-through type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) - when a directory is created/removed, a uevent should happen declaring what type of device was created/removed. So you'll be removing: /sys/devices/system/node/node?/meminfo then? along with: /sys/devices/system/node/node?/distance /sys/devices/system/node/node?/numastat and all the other things that violate the rules? I would love to do that :) And that goes to show how trying to enforce these kinds of rules is damm hard. Things slip by that I never notice because they are only for odd types of hardware :) Is there no way to enforce that in the sysfs interface? (Haven't looked, I'll admit). (which I do agree with ... I just don't think sysfs works for performance stats as we've discussed several times before ;-)) I agree that this doesn't work too, but also that if it's really needed, it can be done, just let us know about it (like /sys/block/BLOCKDEV/stat) OK. Would be nice if we could get rid of /sys/devices/system at some point, which seems like a fairly crazy path, but still ;-) M. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, Oct 10, 2007 at 11:26:32AM -0700, Martin Bligh wrote: Greg KH wrote: On Wed, Oct 10, 2007 at 10:24:43AM -0700, Martin Bligh wrote: The rules for sysfs files are the following: - one value, in text format, per file. - no action apon open/close - binary files are only allowed for pass-through type files that the kernel does not touch (like for firmware and pci config space) - directories should be associated with a kobject where it makes sense (no nesting deep subdirectories without a kobject present) - when a directory is created/removed, a uevent should happen declaring what type of device was created/removed. So you'll be removing: /sys/devices/system/node/node?/meminfo then? along with: /sys/devices/system/node/node?/distance /sys/devices/system/node/node?/numastat and all the other things that violate the rules? I would love to do that :) And that goes to show how trying to enforce these kinds of rules is damm hard. Things slip by that I never notice because they are only for odd types of hardware :) Is there no way to enforce that in the sysfs interface? (Haven't looked, I'll admit). Hm, we could parse the buffer from the response and complain if we notice spaces in it :) (which I do agree with ... I just don't think sysfs works for performance stats as we've discussed several times before ;-)) I agree that this doesn't work too, but also that if it's really needed, it can be done, just let us know about it (like /sys/block/BLOCKDEV/stat) OK. Would be nice if we could get rid of /sys/devices/system at some point, which seems like a fairly crazy path, but still ;-) I agree, getting rid of the sysdev stuff is on the list of things I want to see changed, that code is not nice... thanks, greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Wed, Oct 10, 2007 at 07:16:48AM -0600, Eric W. Biederman wrote: Greg KH [EMAIL PROTECTED] writes: On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: Greg KH [EMAIL PROTECTED] writes: Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Um, no, that's not going to happen. /dev/sda will _always_ have the same major:minor number, as defined by the LSB spec. You can not break that at all. So while you might not want to show all mounts /sys/devices/block/sda/ the ones that you do, will all have the LSB defined major:minor number assigned to it. Hmm. If that is in the LSB it must come from Documentation/devices.txt Yes, that is the requirement. I'm not after changing the user visible major/minor assignments. Oh, I misunderstood what you wrote above then. My above sentence is slightly misleading. That should have been. I am not after changing the device name to major:minor assignments as specified in Documentation/devices.txt. So within a single device namespace everything is normal and as it always has been. Weirdness only ensues when you look across device namespaces. Let me see if a concrete example will help. Suppose I have have a SAN with two disks: disk-1 and disk-2. I have two machines A and B. On machine A I get the mapping: sda - disk-1, sdb -disk-2. On machine B I wind up with a different probe order so I get the mapping: sda - disk-2 sdb -disk-1. Ok. To be very clear by sda I mean the block device with major 8 and minor 0, and by sdb I mean the block device with major 8 and minor 16. Ok. So I decide I want an environment on machine B that looks just like the environment on machine A, so I can bring transfer over a running program or whatever. So I run around looking at UUID labels and what not and I discover that the machine B knows disk-1 as sdb and that machine A knows disk-1 as sda. So I want to say: /sys/devices/block/sdb show up in this other device namespace as /sys/devices/block/sda. Ah, but if you do that then the other device namespace would have /sys/devices/block/sda/dev be 8:16, right? No. The other device namespace I would construct on machine B to look just like the device namespace that existed on machine A. Making /sys/devices/block/sda would still be 8:0. So to be very clear on machine B when talking about disk-1 I would have. initial device namespace: /sys/devices/block/sdb /sys/devices/block/sdb/dev 8:16 other device namespace: /sys/devices/block/sda /sys/devices/block/sda/dev 8:0 Similarly on machine B when talking about disk-2 I would have. initial device namespace: /sys/devices/block/sda /sys/devices/block/sda/dev 8:0 other device namespace: /sys/devices/block/sdb /sys/devices/block/sdb/dev 8:16 So between the two devices namespaces on machine B the two disks would exchange their user visible identities. Ah, ok, that makes more sense. And seems quite difficult to do, good luck with that :) greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH [EMAIL PROTECTED] writes: Ah, ok, that makes more sense. And seems quite difficult to do, good luck with that :) Thanks. At least now all I have to do is worry about the details when we get that far instead of selling the big picture... My gut feel is that sysfs is probably the hardest part to deal with, and maybe half of the problem. Just intercepting the lookup by device number is fairly simple, I think there is one spot for block devices and another for character devices. Anyway once the network namespace support is in with any luck that will have solved half the sysfs problem. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, Oct 09, 2007 at 04:20:39PM -0700, Roland Dreier wrote: > > > - relaying implementation details to userspace so that they cannot be > > > easily changed. We would need to allow kobjects not showing up in sysfs > > > and making symlinks a sysfs facility not relying on kobjects to help > > > there. > > > > Huh? Why would you want a kobject to not show up in sysfs? > > I think the text you replied to explained it perfectly: becase you > don't want internal implementation details to be exposed the userspace > and become an unchangeable part of the kernel/userspace interface. But a kobject represents "something" in the kernel. If it's there, then it shows up in sysfs. But if it isn't, or it changes somehow, then it no longer is in sysfs, which is fine, and your userspace tools have to be able to handle that by virtue of the rules of how to use sysfs from userspace. thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
> > - relaying implementation details to userspace so that they cannot be > > easily changed. We would need to allow kobjects not showing up in sysfs > > and making symlinks a sysfs facility not relying on kobjects to help > > there. > > Huh? Why would you want a kobject to not show up in sysfs? I think the text you replied to explained it perfectly: becase you don't want internal implementation details to be exposed the userspace and become an unchangeable part of the kernel/userspace interface. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, Oct 05, 2007 at 06:44:04AM -0600, Eric W. Biederman wrote: > Greg KH <[EMAIL PROTECTED]> writes: > > > I would be interested in seeing what your patches look like. > > Sure. > > > I don't > > think that we should take any more sysfs changes for 2.6.24 as we do > > have a lot of them right now, and I don't think that Tejun and I agree > > on the future direction of the outstanding ones just yet. > > Sounds reasonable. > > > But I don't think that your multiple-mount patches could make it into > > .24, unless .23 is still weeks away. > > Well I have posted them all earlier. At this point I it makes most > sense to wait until after the big merge happen and every rebases on > top of that. Then everyone will have network namespace support and > it is easier to look through all of the patches. Especially since > it looks like the merge window will open any day now. > > I will quickly recap the essence of what I am looking at: > On directories of interest I tag all of their directory > entries with which namespace they belong to. On a mount > of sysfs I remember which namespace we were in when we > mounted sysfs. The I filter readdir and lookup based upon > the namespace I captured at mount time. I do my best > to generalize it so that the logic can work for different > namespaces. Ok, I have no objection to that. Let's wait for 2.6.24 to settle down :) thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, Oct 05, 2007 at 05:00:48PM +0900, Tejun Heo wrote: > Hello, Greg. > > I think this definitely needs more discussion, so here we go... heh :) > Greg KH wrote: > >> 1. What is a kobject? > [--snip--] > >> The functionality served by kobject can be broken down into the > >> following two. > >> > >> F-a. To serve as an entity both subsystems can share lifespan > >> management. ie. both subsystems reference count on kobject. > >> > >> F-b. To serve as an entity both subsystems can base their internal > >> representation on. (base object in OO term). > > > > Yes, those are two functions, I can agree with. > > > [--snip--] > >> 3. Where does that leave kobject? > >> > >> If you combine #1 and #2, both functionalities become questionable. > >> > >> F-a. sysfs no longer plays role in lifespan management of driver model > >> object. This functionality is exactly what's killed by #2. > > > > Yes, but a kobject is still needed internally for the lifespan > > management. > > Yes, exactly - "internally" to the driver model (or drivers which ride > along). To sysfs, it has no function other than being an opaque token. I agree. > [--snip--] > >> IMHO, both L-a and L-b contribute only to obfuscation of the driver > >> model and sysfs. > > > > No. I think you are missing a number of things that kobjects provide > > and allow: > > - a structurual heirachy of devices. Combine kobjects with > > ksets and ktypes, and you have a very powerful system to > > categorize objects and their representation to each other. > > Yes, which only needs to be used _inside_ the driver model > implementation proper. And for kobjects too. They need a way to show categories and heirachy. > > - a consistant and easy interface to userspace through > > uevents/hotplug of the creation and removal of these objects. > > This keeps the different parts of the kernel that need this > > interface from having to create it every time on their own. > > Things can be much easier than now. Also, the above paragraph is > inconsistent with the rest of your argument or am I misunderstanding > what you mean by the above paragraph? I think you must be misunderstanding what I mean here. In the past, to add something like "userspace notification of hotplug" to different kernel subsystems, we had to add it all over the place. Now, with a unified way to represent objects in the kernel (kobjects and then 'struct device') we only have to add the logic in one place for the core code, and all subsystems have automatic access to it. That's one of the main benefits of this core code. > > - a way to easily create and export attributes in sysfs > > automatically. > > This is and should be the function of the driver model not kobjects. > Removing kobject from the interface doesn't change anything about this. No, kobjects need to do this, not just the driver model. What about things that do not currently use the driver model (like block devices). Are you going to tell them they can't have attributes? :) > > - a way to provide working reference counting for a variety of > > different objects. > > To me, this just feels wrong and does more harm than it helps. I really > think we shouldn't have multi-role flash light at the core of our driver > model (inside driver model proper, no problem, but not as exported > interface). Well, that's what struct kref does, as Cornelia pointed out. That's even lower down than a kobject. > > All of those are still needed for the kernel. > > For #1 and #3, I agree if you limit the scope to driver model proper but > I am not arguing kobject and all its friends should be abolished. I'm > arguing that there is no reason to export it as API because it doesn't > add any value exported. I think the block layer would disagree with you :) > For #4, I don't know. This can be a matter of taste but I don't think > #4 alone stands as justification for kobject as external API. Sorry, struct kref does that. You want a kobject to show hierarchy and classification of types. > For #2, I might be misundertanding. Please elaborate if I am. For > uevent issue, I'll talk about more later. > > So, I honestly don't think the above four arguments successfully counter > the original arguments. If I'm missing something, feel free to hammer > me into enlightenment. :-) The "original" argument was that I don't want to allow users of sysfs who are not using a struct kobject. Perhaps we should just focus on that issue instead of debating the relative merits of kobject and struct device for right now. > >> 4. So? > >> > >> From #3, as kobject no longer serves any valid purpose to sysfs, it's > >> natural conclusion to try to remove kobject from sysfs, which of course > >> brings up the question of conversion cost. > > > > I don't mind the removal of kobjects from sysfs in order to make sysfs > > and kobjects work better/simpler. However the majority of the
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: > Greg KH <[EMAIL PROTECTED]> writes: > > > >> Also fun is that the dev file implementation needs to be able to > >> report different major:minor numbers based on which mount of > >> sysfs we are dealing with. > > > > Um, no, that's not going to happen. /dev/sda will _always_ have the > > same major:minor number, as defined by the LSB spec. You can not break > > that at all. So while you might not want to show all mounts > > /sys/devices/block/sda/ the ones that you do, will all have the LSB > > defined major:minor number assigned to it. > > Hmm. If that is in the LSB it must come from > Documentation/devices.txt Yes, that is the requirement. > I'm not after changing the user visible major/minor assignments. Oh, I misunderstood what you wrote above then. > Let me see if a concrete example will help. Suppose I have > have a SAN with two disks: disk-1 and disk-2. I have > two machines A and B. On machine A I get the mapping: > sda -> disk-1, sdb ->disk-2. On machine B I wind up with > a different probe order so I get the mapping: sda -> disk-2 > sdb ->disk-1. Ok. > To be very clear by sda I mean the block device with major 8 and > minor 0, and by sdb I mean the block device with major 8 and minor > 16. Ok. > So I decide I want an environment on machine B that looks just > like the environment on machine A, so I can bring transfer over > a running program or whatever. So I run around looking at UUID > labels and what not and I discover that the machine B knows disk-1 as > sdb and that machine A knows disk-1 as sda. So I want to say: > /sys/devices/block/sdb show up in this other device namespace as > /sys/devices/block/sda. Ah, but if you do that then the "other" device namespace would have /sys/devices/block/sda/dev be 8:16, right? And that is not valid as sda for that namespace must always map to the device with a 8:0 major:minor as per the LSB spec. So, no, that's not going to be allowed, sorry. thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, Oct 09, 2007 at 11:29:01AM +0200, Cornelia Huck wrote: > On Fri, 05 Oct 2007 17:00:48 +0900, > Tejun Heo <[EMAIL PROTECTED]> wrote: > > > I think this definitely needs more discussion, so here we go... > > I agree, so I'll give my 0.02 ? here... > > > > Greg KH wrote: > > >> 1. What is a kobject? > > [--snip--] > > >> The functionality served by kobject can be broken down into the > > >> following two. > > >> > > >> F-a. To serve as an entity both subsystems can share lifespan > > >> management. ie. both subsystems reference count on kobject. > > >> > > >> F-b. To serve as an entity both subsystems can base their internal > > >> representation on. (base object in OO term). > > > > > > Yes, those are two functions, I can agree with. > > I think that's the heart of the question: We first need to agree what > use the different components should have. > > (a) The driver model > > The driver model serves as a unified layer for all devices managed by > the kernel, organized in trees, and the drivers handling them. This > includes busses, matching of devices and drivers, attributes and so on. > Userspace expects to see these devices, drivers, busses and attributes > by looking under /sys/devices/. /sys/class/ and /sys/bus/ provide > additional views on this data. > > (b) kobjects, ktypes, ksets > > kobjects provide a mechanism to arrange kernel objects into a tree-like > structure. ktypes and ksets are mechanisms to further order these > objects. Changes in the kobject hierarchy are communicated to userspace > via uevents. > > The driver core is implemented using this infrastructure. > > (c) krefs > > krefs provide a generic reference counting mechanism. > > The kobject infrastructure uses krefs for its reference counting needs. > > (d) sysfs > > sysfs is a virtual filesystem. It exports information on kernel objects > to user space. (IMO, that's the key: sysfs is userspace representation.) Ok, I agree with all of the above :) > That said, it is logical that kobjects are made visible to userspace > via sysfs. If someone is trying to make things show up in sysfs and has > to jump through hoops to cook up kobjects, they're probably using the > wrong infrastructure. I agree. > There are two big problems with the tight coupling of sysfs and kobjects: > > - lifetime rules; but this fortunately hugely improved with the > previous patches :) Yes, I think that's pretty much fixed now. > - relaying implementation details to userspace so that they cannot be > easily changed. We would need to allow kobjects not showing up in sysfs > and making symlinks a sysfs facility not relying on kobjects to help > there. Huh? Why would you want a kobject to not show up in sysfs? And yes, I agree we could use some more "automatic" help in regards to symlinks in sysfs when we change kobjects around. That would make things simpler for the kobject core. > > [--snip--] > > >> 3. Where does that leave kobject? > > >> > > >> If you combine #1 and #2, both functionalities become questionable. > > >> > > >> F-a. sysfs no longer plays role in lifespan management of driver model > > >> object. This functionality is exactly what's killed by #2. > > > > > > Yes, but a kobject is still needed internally for the lifespan > > > management. > > > > Yes, exactly - "internally" to the driver model (or drivers which ride > > along). To sysfs, it has no function other than being an opaque token. > > But krefs are used for kobject reference counting, or am I > misunderstanding here? > > > > > [--snip--] > > >> IMHO, both L-a and L-b contribute only to obfuscation of the driver > > >> model and sysfs. > > > > > > No. I think you are missing a number of things that kobjects provide > > > and allow: > > > - a structurual heirachy of devices. Combine kobjects with > > > ksets and ktypes, and you have a very powerful system to > > > categorize objects and their representation to each other. > > > > Yes, which only needs to be used _inside_ the driver model > > implementation proper. > > There are use cases outside of the driver model prober where you may > want to use kobject for hierarchy. agreed. > > > - a consistant and easy interface to userspace through > > > uevents/hotplug of the creation and removal of these objects. > > > This keeps the different parts of the kernel that need this > > > interface from having to create it every time on their own. > > > > Things can be much easier than now. Also, the above paragraph is > > inconsistent with the rest of your argument or am I misunderstanding > > what you mean by the above paragraph? > > I see uevents as a notifier for changes in the kobject hierarchy, so > they belong to that layer. However, the layering between kobjects and > sysfs (path names etc.) could probably be made cleaner. also agreed. > > > - a way to easily create and export attributes in sysfs > > > automatically. > > > > This is and should be the function
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, 05 Oct 2007 17:00:48 +0900, Tejun Heo <[EMAIL PROTECTED]> wrote: > I think this definitely needs more discussion, so here we go... I agree, so I'll give my 0.02 € here... > > Greg KH wrote: > >> 1. What is a kobject? > [--snip--] > >> The functionality served by kobject can be broken down into the > >> following two. > >> > >> F-a. To serve as an entity both subsystems can share lifespan > >> management. ie. both subsystems reference count on kobject. > >> > >> F-b. To serve as an entity both subsystems can base their internal > >> representation on. (base object in OO term). > > > > Yes, those are two functions, I can agree with. I think that's the heart of the question: We first need to agree what use the different components should have. (a) The driver model The driver model serves as a unified layer for all devices managed by the kernel, organized in trees, and the drivers handling them. This includes busses, matching of devices and drivers, attributes and so on. Userspace expects to see these devices, drivers, busses and attributes by looking under /sys/devices/. /sys/class/ and /sys/bus/ provide additional views on this data. (b) kobjects, ktypes, ksets kobjects provide a mechanism to arrange kernel objects into a tree-like structure. ktypes and ksets are mechanisms to further order these objects. Changes in the kobject hierarchy are communicated to userspace via uevents. The driver core is implemented using this infrastructure. (c) krefs krefs provide a generic reference counting mechanism. The kobject infrastructure uses krefs for its reference counting needs. (d) sysfs sysfs is a virtual filesystem. It exports information on kernel objects to user space. (IMO, that's the key: sysfs is userspace representation.) That said, it is logical that kobjects are made visible to userspace via sysfs. If someone is trying to make things show up in sysfs and has to jump through hoops to cook up kobjects, they're probably using the wrong infrastructure. There are two big problems with the tight coupling of sysfs and kobjects: - lifetime rules; but this fortunately hugely improved with the previous patches :) - relaying implementation details to userspace so that they cannot be easily changed. We would need to allow kobjects not showing up in sysfs and making symlinks a sysfs facility not relying on kobjects to help there. > > > [--snip--] > >> 3. Where does that leave kobject? > >> > >> If you combine #1 and #2, both functionalities become questionable. > >> > >> F-a. sysfs no longer plays role in lifespan management of driver model > >> object. This functionality is exactly what's killed by #2. > > > > Yes, but a kobject is still needed internally for the lifespan > > management. > > Yes, exactly - "internally" to the driver model (or drivers which ride > along). To sysfs, it has no function other than being an opaque token. But krefs are used for kobject reference counting, or am I misunderstanding here? > > [--snip--] > >> IMHO, both L-a and L-b contribute only to obfuscation of the driver > >> model and sysfs. > > > > No. I think you are missing a number of things that kobjects provide > > and allow: > > - a structurual heirachy of devices. Combine kobjects with > > ksets and ktypes, and you have a very powerful system to > > categorize objects and their representation to each other. > > Yes, which only needs to be used _inside_ the driver model > implementation proper. There are use cases outside of the driver model prober where you may want to use kobject for hierarchy. > > > - a consistant and easy interface to userspace through > > uevents/hotplug of the creation and removal of these objects. > > This keeps the different parts of the kernel that need this > > interface from having to create it every time on their own. > > Things can be much easier than now. Also, the above paragraph is > inconsistent with the rest of your argument or am I misunderstanding > what you mean by the above paragraph? I see uevents as a notifier for changes in the kobject hierarchy, so they belong to that layer. However, the layering between kobjects and sysfs (path names etc.) could probably be made cleaner. > > > - a way to easily create and export attributes in sysfs > > automatically. > > This is and should be the function of the driver model not kobjects. I agree, attributes should belong to the driver model. > Removing kobject from the interface doesn't change anything about this. Hm. Currently you have a file<->attribute correlation. This would change if you allow non-attribute files. > > > - a way to provide working reference counting for a variety of > > different objects. > > To me, this just feels wrong and does more harm than it helps. I really > think we shouldn't have multi-role flash light at the core of our driver > model (inside driver model proper, no problem, but not as exported > interface).
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, 05 Oct 2007 17:00:48 +0900, Tejun Heo [EMAIL PROTECTED] wrote: I think this definitely needs more discussion, so here we go... I agree, so I'll give my 0.02 € here... Greg KH wrote: 1. What is a kobject? [--snip--] The functionality served by kobject can be broken down into the following two. F-a. To serve as an entity both subsystems can share lifespan management. ie. both subsystems reference count on kobject. F-b. To serve as an entity both subsystems can base their internal representation on. (base object in OO term). Yes, those are two functions, I can agree with. I think that's the heart of the question: We first need to agree what use the different components should have. (a) The driver model The driver model serves as a unified layer for all devices managed by the kernel, organized in trees, and the drivers handling them. This includes busses, matching of devices and drivers, attributes and so on. Userspace expects to see these devices, drivers, busses and attributes by looking under /sys/devices/. /sys/class/ and /sys/bus/ provide additional views on this data. (b) kobjects, ktypes, ksets kobjects provide a mechanism to arrange kernel objects into a tree-like structure. ktypes and ksets are mechanisms to further order these objects. Changes in the kobject hierarchy are communicated to userspace via uevents. The driver core is implemented using this infrastructure. (c) krefs krefs provide a generic reference counting mechanism. The kobject infrastructure uses krefs for its reference counting needs. (d) sysfs sysfs is a virtual filesystem. It exports information on kernel objects to user space. (IMO, that's the key: sysfs is userspace representation.) That said, it is logical that kobjects are made visible to userspace via sysfs. If someone is trying to make things show up in sysfs and has to jump through hoops to cook up kobjects, they're probably using the wrong infrastructure. There are two big problems with the tight coupling of sysfs and kobjects: - lifetime rules; but this fortunately hugely improved with the previous patches :) - relaying implementation details to userspace so that they cannot be easily changed. We would need to allow kobjects not showing up in sysfs and making symlinks a sysfs facility not relying on kobjects to help there. [--snip--] 3. Where does that leave kobject? If you combine #1 and #2, both functionalities become questionable. F-a. sysfs no longer plays role in lifespan management of driver model object. This functionality is exactly what's killed by #2. Yes, but a kobject is still needed internally for the lifespan management. Yes, exactly - internally to the driver model (or drivers which ride along). To sysfs, it has no function other than being an opaque token. But krefs are used for kobject reference counting, or am I misunderstanding here? [--snip--] IMHO, both L-a and L-b contribute only to obfuscation of the driver model and sysfs. No. I think you are missing a number of things that kobjects provide and allow: - a structurual heirachy of devices. Combine kobjects with ksets and ktypes, and you have a very powerful system to categorize objects and their representation to each other. Yes, which only needs to be used _inside_ the driver model implementation proper. There are use cases outside of the driver model prober where you may want to use kobject for hierarchy. - a consistant and easy interface to userspace through uevents/hotplug of the creation and removal of these objects. This keeps the different parts of the kernel that need this interface from having to create it every time on their own. Things can be much easier than now. Also, the above paragraph is inconsistent with the rest of your argument or am I misunderstanding what you mean by the above paragraph? I see uevents as a notifier for changes in the kobject hierarchy, so they belong to that layer. However, the layering between kobjects and sysfs (path names etc.) could probably be made cleaner. - a way to easily create and export attributes in sysfs automatically. This is and should be the function of the driver model not kobjects. I agree, attributes should belong to the driver model. Removing kobject from the interface doesn't change anything about this. Hm. Currently you have a file-attribute correlation. This would change if you allow non-attribute files. - a way to provide working reference counting for a variety of different objects. To me, this just feels wrong and does more harm than it helps. I really think we shouldn't have multi-role flash light at the core of our driver model (inside driver model proper, no problem, but not as exported interface). And I still think that this is the purpose of krefs :) All of those are still needed for the kernel. For #1 and #3,
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, Oct 09, 2007 at 11:29:01AM +0200, Cornelia Huck wrote: On Fri, 05 Oct 2007 17:00:48 +0900, Tejun Heo [EMAIL PROTECTED] wrote: I think this definitely needs more discussion, so here we go... I agree, so I'll give my 0.02 ? here... Greg KH wrote: 1. What is a kobject? [--snip--] The functionality served by kobject can be broken down into the following two. F-a. To serve as an entity both subsystems can share lifespan management. ie. both subsystems reference count on kobject. F-b. To serve as an entity both subsystems can base their internal representation on. (base object in OO term). Yes, those are two functions, I can agree with. I think that's the heart of the question: We first need to agree what use the different components should have. (a) The driver model The driver model serves as a unified layer for all devices managed by the kernel, organized in trees, and the drivers handling them. This includes busses, matching of devices and drivers, attributes and so on. Userspace expects to see these devices, drivers, busses and attributes by looking under /sys/devices/. /sys/class/ and /sys/bus/ provide additional views on this data. (b) kobjects, ktypes, ksets kobjects provide a mechanism to arrange kernel objects into a tree-like structure. ktypes and ksets are mechanisms to further order these objects. Changes in the kobject hierarchy are communicated to userspace via uevents. The driver core is implemented using this infrastructure. (c) krefs krefs provide a generic reference counting mechanism. The kobject infrastructure uses krefs for its reference counting needs. (d) sysfs sysfs is a virtual filesystem. It exports information on kernel objects to user space. (IMO, that's the key: sysfs is userspace representation.) Ok, I agree with all of the above :) That said, it is logical that kobjects are made visible to userspace via sysfs. If someone is trying to make things show up in sysfs and has to jump through hoops to cook up kobjects, they're probably using the wrong infrastructure. I agree. There are two big problems with the tight coupling of sysfs and kobjects: - lifetime rules; but this fortunately hugely improved with the previous patches :) Yes, I think that's pretty much fixed now. - relaying implementation details to userspace so that they cannot be easily changed. We would need to allow kobjects not showing up in sysfs and making symlinks a sysfs facility not relying on kobjects to help there. Huh? Why would you want a kobject to not show up in sysfs? And yes, I agree we could use some more automatic help in regards to symlinks in sysfs when we change kobjects around. That would make things simpler for the kobject core. [--snip--] 3. Where does that leave kobject? If you combine #1 and #2, both functionalities become questionable. F-a. sysfs no longer plays role in lifespan management of driver model object. This functionality is exactly what's killed by #2. Yes, but a kobject is still needed internally for the lifespan management. Yes, exactly - internally to the driver model (or drivers which ride along). To sysfs, it has no function other than being an opaque token. But krefs are used for kobject reference counting, or am I misunderstanding here? [--snip--] IMHO, both L-a and L-b contribute only to obfuscation of the driver model and sysfs. No. I think you are missing a number of things that kobjects provide and allow: - a structurual heirachy of devices. Combine kobjects with ksets and ktypes, and you have a very powerful system to categorize objects and their representation to each other. Yes, which only needs to be used _inside_ the driver model implementation proper. There are use cases outside of the driver model prober where you may want to use kobject for hierarchy. agreed. - a consistant and easy interface to userspace through uevents/hotplug of the creation and removal of these objects. This keeps the different parts of the kernel that need this interface from having to create it every time on their own. Things can be much easier than now. Also, the above paragraph is inconsistent with the rest of your argument or am I misunderstanding what you mean by the above paragraph? I see uevents as a notifier for changes in the kobject hierarchy, so they belong to that layer. However, the layering between kobjects and sysfs (path names etc.) could probably be made cleaner. also agreed. - a way to easily create and export attributes in sysfs automatically. This is and should be the function of the driver model not kobjects. I agree, attributes should belong to the driver model. Removing kobject from the interface doesn't change anything about this. Hm. Currently you have a file-attribute correlation.
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, Oct 05, 2007 at 05:00:48PM +0900, Tejun Heo wrote: Hello, Greg. I think this definitely needs more discussion, so here we go... heh :) Greg KH wrote: 1. What is a kobject? [--snip--] The functionality served by kobject can be broken down into the following two. F-a. To serve as an entity both subsystems can share lifespan management. ie. both subsystems reference count on kobject. F-b. To serve as an entity both subsystems can base their internal representation on. (base object in OO term). Yes, those are two functions, I can agree with. [--snip--] 3. Where does that leave kobject? If you combine #1 and #2, both functionalities become questionable. F-a. sysfs no longer plays role in lifespan management of driver model object. This functionality is exactly what's killed by #2. Yes, but a kobject is still needed internally for the lifespan management. Yes, exactly - internally to the driver model (or drivers which ride along). To sysfs, it has no function other than being an opaque token. I agree. [--snip--] IMHO, both L-a and L-b contribute only to obfuscation of the driver model and sysfs. No. I think you are missing a number of things that kobjects provide and allow: - a structurual heirachy of devices. Combine kobjects with ksets and ktypes, and you have a very powerful system to categorize objects and their representation to each other. Yes, which only needs to be used _inside_ the driver model implementation proper. And for kobjects too. They need a way to show categories and heirachy. - a consistant and easy interface to userspace through uevents/hotplug of the creation and removal of these objects. This keeps the different parts of the kernel that need this interface from having to create it every time on their own. Things can be much easier than now. Also, the above paragraph is inconsistent with the rest of your argument or am I misunderstanding what you mean by the above paragraph? I think you must be misunderstanding what I mean here. In the past, to add something like userspace notification of hotplug to different kernel subsystems, we had to add it all over the place. Now, with a unified way to represent objects in the kernel (kobjects and then 'struct device') we only have to add the logic in one place for the core code, and all subsystems have automatic access to it. That's one of the main benefits of this core code. - a way to easily create and export attributes in sysfs automatically. This is and should be the function of the driver model not kobjects. Removing kobject from the interface doesn't change anything about this. No, kobjects need to do this, not just the driver model. What about things that do not currently use the driver model (like block devices). Are you going to tell them they can't have attributes? :) - a way to provide working reference counting for a variety of different objects. To me, this just feels wrong and does more harm than it helps. I really think we shouldn't have multi-role flash light at the core of our driver model (inside driver model proper, no problem, but not as exported interface). Well, that's what struct kref does, as Cornelia pointed out. That's even lower down than a kobject. All of those are still needed for the kernel. For #1 and #3, I agree if you limit the scope to driver model proper but I am not arguing kobject and all its friends should be abolished. I'm arguing that there is no reason to export it as API because it doesn't add any value exported. I think the block layer would disagree with you :) For #4, I don't know. This can be a matter of taste but I don't think #4 alone stands as justification for kobject as external API. Sorry, struct kref does that. You want a kobject to show hierarchy and classification of types. For #2, I might be misundertanding. Please elaborate if I am. For uevent issue, I'll talk about more later. So, I honestly don't think the above four arguments successfully counter the original arguments. If I'm missing something, feel free to hammer me into enlightenment. :-) The original argument was that I don't want to allow users of sysfs who are not using a struct kobject. Perhaps we should just focus on that issue instead of debating the relative merits of kobject and struct device for right now. 4. So? From #3, as kobject no longer serves any valid purpose to sysfs, it's natural conclusion to try to remove kobject from sysfs, which of course brings up the question of conversion cost. I don't mind the removal of kobjects from sysfs in order to make sysfs and kobjects work better/simpler. However the majority of the patches you created to do this end up with more code overall, and are of no benifit to the current users of sysfs and kobjects in the kernel. 95+% of
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, Oct 05, 2007 at 06:12:41AM -0600, Eric W. Biederman wrote: Greg KH [EMAIL PROTECTED] writes: Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Um, no, that's not going to happen. /dev/sda will _always_ have the same major:minor number, as defined by the LSB spec. You can not break that at all. So while you might not want to show all mounts /sys/devices/block/sda/ the ones that you do, will all have the LSB defined major:minor number assigned to it. Hmm. If that is in the LSB it must come from Documentation/devices.txt Yes, that is the requirement. I'm not after changing the user visible major/minor assignments. Oh, I misunderstood what you wrote above then. Let me see if a concrete example will help. Suppose I have have a SAN with two disks: disk-1 and disk-2. I have two machines A and B. On machine A I get the mapping: sda - disk-1, sdb -disk-2. On machine B I wind up with a different probe order so I get the mapping: sda - disk-2 sdb -disk-1. Ok. To be very clear by sda I mean the block device with major 8 and minor 0, and by sdb I mean the block device with major 8 and minor 16. Ok. So I decide I want an environment on machine B that looks just like the environment on machine A, so I can bring transfer over a running program or whatever. So I run around looking at UUID labels and what not and I discover that the machine B knows disk-1 as sdb and that machine A knows disk-1 as sda. So I want to say: /sys/devices/block/sdb show up in this other device namespace as /sys/devices/block/sda. Ah, but if you do that then the other device namespace would have /sys/devices/block/sda/dev be 8:16, right? And that is not valid as sda for that namespace must always map to the device with a 8:0 major:minor as per the LSB spec. So, no, that's not going to be allowed, sorry. thanks, greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Fri, Oct 05, 2007 at 06:44:04AM -0600, Eric W. Biederman wrote: Greg KH [EMAIL PROTECTED] writes: I would be interested in seeing what your patches look like. Sure. I don't think that we should take any more sysfs changes for 2.6.24 as we do have a lot of them right now, and I don't think that Tejun and I agree on the future direction of the outstanding ones just yet. Sounds reasonable. But I don't think that your multiple-mount patches could make it into .24, unless .23 is still weeks away. Well I have posted them all earlier. At this point I it makes most sense to wait until after the big merge happen and every rebases on top of that. Then everyone will have network namespace support and it is easier to look through all of the patches. Especially since it looks like the merge window will open any day now. I will quickly recap the essence of what I am looking at: On directories of interest I tag all of their directory entries with which namespace they belong to. On a mount of sysfs I remember which namespace we were in when we mounted sysfs. The I filter readdir and lookup based upon the namespace I captured at mount time. I do my best to generalize it so that the logic can work for different namespaces. Ok, I have no objection to that. Let's wait for 2.6.24 to settle down :) thanks, greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
- relaying implementation details to userspace so that they cannot be easily changed. We would need to allow kobjects not showing up in sysfs and making symlinks a sysfs facility not relying on kobjects to help there. Huh? Why would you want a kobject to not show up in sysfs? I think the text you replied to explained it perfectly: becase you don't want internal implementation details to be exposed the userspace and become an unchangeable part of the kernel/userspace interface. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Tue, Oct 09, 2007 at 04:20:39PM -0700, Roland Dreier wrote: - relaying implementation details to userspace so that they cannot be easily changed. We would need to allow kobjects not showing up in sysfs and making symlinks a sysfs facility not relying on kobjects to help there. Huh? Why would you want a kobject to not show up in sysfs? I think the text you replied to explained it perfectly: becase you don't want internal implementation details to be exposed the userspace and become an unchangeable part of the kernel/userspace interface. But a kobject represents something in the kernel. If it's there, then it shows up in sysfs. But if it isn't, or it changes somehow, then it no longer is in sysfs, which is fine, and your userspace tools have to be able to handle that by virtue of the rules of how to use sysfs from userspace. thanks, greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Devel] Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Kirill Korotaev <[EMAIL PROTECTED]> writes: > Imho environments to be migratable should have no direct access to the > devices. > You can use any of stacked virtual filesystems to hide real > device from container. > You will have problems much bigger than this one otherwise > (imagine access to video, sound etc.) What I am primarily concern about is when you can make the case that the hardware we are talking is present before and after the migration. When you are directly accessing a device. For times when it makes sense to directly access hardware in a container (think infiniband OS-bypass NICs). We need to tell user space that the device was unplugged and another one was plugged in. If user space can cope with that things should continue to work. There are some very specific cases that we can support: - Stateless devices like /dev/zero and dev/random. - Virtual devices like ttys, ramdisks, loop devices - Remote block devices like SCSI disks on a san, iSCSI, nbd, ATAoE. - Local pseudo block devices like the backing devices for virtual filesystems. There are very specific limits in which this can work and be useable, and I don't claim to have looked at all of the details, but for the block device case in particular we export the block device number to user space in stat. There are some common applications which do memorize stat data for files things like: git, incremental backup software, and intrusion detection software. Frankly the times when this matters is rare enough I don't put a big priority on getting this done quickly. However a strong case has been made for all of this filtering so we can run things like udev in a container. Initially I only expect stateless character devices and ttys to be allowed in a namespace, and I don't have a clue what device number we will use in st_dev for stat in the case our block device isn't in the user space interface. I just know that it sounds like where we want to be eventually and thinking about it now isn't a problem. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Devel] Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Eric W. Biederman wrote: > Greg KH <[EMAIL PROTECTED]> writes: > >>> Also fun is that the dev file implementation needs to be able to >>> report different major:minor numbers based on which mount of >>> sysfs we are dealing with. >> >>Um, no, that's not going to happen. /dev/sda will _always_ have the >>same major:minor number, as defined by the LSB spec. You can not break >>that at all. So while you might not want to show all mounts >>/sys/devices/block/sda/ the ones that you do, will all have the LSB >>defined major:minor number assigned to it. > > > Hmm. If that is in the LSB it must come from > Documentation/devices.txt I'm not after changing the user > visible major/minor assignments. > > Let me see if a concrete example will help. Suppose I have > have a SAN with two disks: disk-1 and disk-2. I have > two machines A and B. On machine A I get the mapping: > sda -> disk-1, sdb ->disk-2. On machine B I wind up with > a different probe order so I get the mapping: sda -> disk-2 > sdb ->disk-1. > > To be very clear by sda I mean the block device with major 8 and > minor 0, and by sdb I mean the block device with major 8 and minor > 16. > > So I decide I want an environment on machine B that looks just > like the environment on machine A, so I can bring transfer over > a running program or whatever. So I run around looking at UUID > labels and what not and I discover that the machine B knows disk-1 as > sdb and that machine A knows disk-1 as sda. So I want to say: > /sys/devices/block/sdb show up in this other device namespace as > /sys/devices/block/sda. Imho environments to be migratable should have no direct access to the devices. You can use any of stacked virtual filesystems to hide real device from container. You will have problems much bigger than this one otherwise (imagine access to video, sound etc.) Kirill - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH <[EMAIL PROTECTED]> writes:
> I would be interested in seeing what your patches look like.
Sure.
> I don't
> think that we should take any more sysfs changes for 2.6.24 as we do
> have a lot of them right now, and I don't think that Tejun and I agree
> on the future direction of the outstanding ones just yet.
Sounds reasonable.
> But I don't think that your multiple-mount patches could make it into
> .24, unless .23 is still weeks away.
Well I have posted them all earlier. At this point I it makes most
sense to wait until after the big merge happen and every rebases on
top of that. Then everyone will have network namespace support and
it is easier to look through all of the patches. Especially since
it looks like the merge window will open any day now.
I will quickly recap the essence of what I am looking at:
On directories of interest I tag all of their directory
entries with which namespace they belong to. On a mount
of sysfs I remember which namespace we were in when we
mounted sysfs. The I filter readdir and lookup based upon
the namespace I captured at mount time. I do my best
to generalize it so that the logic can work for different
namespaces.
Currently the heart of the patch from the network namespace
is below (I sniped the part that does the capture at mount time).
Basically the interface to users of this functionality is just
providing some way to go from a super block or a kobject to
the tag sysfs is using to filter things.
So I get one sysfs_dirent tree, but each super_block has it's
own tree of dcache entries.
Everything else is pretty much details in checking and propagating
the tags into the appropriate places.
Eric
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 5adfdc2..a300f6e 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -435,6 +437,23 @@ static void netdev_release(struct device *d)
kfree((char *)dev - dev->padded);
}
+static const void *net_sb_tag(struct sysfs_tag_info *info)
+{
+ return info->net_ns;
+}
+
+static const void *net_kobject_tag(struct kobject *kobj)
+{
+ struct net_device *dev;
+ dev = container_of(kobj, struct net_device, dev.kobj);
+ return dev->nd_net;
+}
+
+static const struct sysfs_tagged_dir_operations net_tagged_dir_operations = {
+ .sb_tag = net_sb_tag,
+ .kobject_tag = net_kobject_tag,
+};
+
static struct class net_class = {
.name = "net",
.dev_release = netdev_release,
@@ -444,6 +463,7 @@ static struct class net_class = {
#ifdef CONFIG_HOTPLUG
.dev_uevent = netdev_uevent,
#endif
+ .tag_ops = _tagged_dir_operations,
};
/* Delete sysfs entries but hold kobject reference until after all
--
1.5.3.rc6.17.g1911
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH <[EMAIL PROTECTED]> writes: > >> Also fun is that the dev file implementation needs to be able to >> report different major:minor numbers based on which mount of >> sysfs we are dealing with. > > Um, no, that's not going to happen. /dev/sda will _always_ have the > same major:minor number, as defined by the LSB spec. You can not break > that at all. So while you might not want to show all mounts > /sys/devices/block/sda/ the ones that you do, will all have the LSB > defined major:minor number assigned to it. Hmm. If that is in the LSB it must come from Documentation/devices.txt I'm not after changing the user visible major/minor assignments. Let me see if a concrete example will help. Suppose I have have a SAN with two disks: disk-1 and disk-2. I have two machines A and B. On machine A I get the mapping: sda -> disk-1, sdb ->disk-2. On machine B I wind up with a different probe order so I get the mapping: sda -> disk-2 sdb ->disk-1. To be very clear by sda I mean the block device with major 8 and minor 0, and by sdb I mean the block device with major 8 and minor 16. So I decide I want an environment on machine B that looks just like the environment on machine A, so I can bring transfer over a running program or whatever. So I run around looking at UUID labels and what not and I discover that the machine B knows disk-1 as sdb and that machine A knows disk-1 as sda. So I want to say: /sys/devices/block/sdb show up in this other device namespace as /sys/devices/block/sda. In that instance a running program won't notice the difference. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Hello, Greg. I think this definitely needs more discussion, so here we go... Greg KH wrote: >> 1. What is a kobject? [--snip--] >> The functionality served by kobject can be broken down into the >> following two. >> >> F-a. To serve as an entity both subsystems can share lifespan >> management. ie. both subsystems reference count on kobject. >> >> F-b. To serve as an entity both subsystems can base their internal >> representation on. (base object in OO term). > > Yes, those are two functions, I can agree with. > [--snip--] >> 3. Where does that leave kobject? >> >> If you combine #1 and #2, both functionalities become questionable. >> >> F-a. sysfs no longer plays role in lifespan management of driver model >> object. This functionality is exactly what's killed by #2. > > Yes, but a kobject is still needed internally for the lifespan > management. Yes, exactly - "internally" to the driver model (or drivers which ride along). To sysfs, it has no function other than being an opaque token. [--snip--] >> IMHO, both L-a and L-b contribute only to obfuscation of the driver >> model and sysfs. > > No. I think you are missing a number of things that kobjects provide > and allow: > - a structurual heirachy of devices. Combine kobjects with > ksets and ktypes, and you have a very powerful system to > categorize objects and their representation to each other. Yes, which only needs to be used _inside_ the driver model implementation proper. > - a consistant and easy interface to userspace through > uevents/hotplug of the creation and removal of these objects. > This keeps the different parts of the kernel that need this > interface from having to create it every time on their own. Things can be much easier than now. Also, the above paragraph is inconsistent with the rest of your argument or am I misunderstanding what you mean by the above paragraph? > - a way to easily create and export attributes in sysfs > automatically. This is and should be the function of the driver model not kobjects. Removing kobject from the interface doesn't change anything about this. > - a way to provide working reference counting for a variety of > different objects. To me, this just feels wrong and does more harm than it helps. I really think we shouldn't have multi-role flash light at the core of our driver model (inside driver model proper, no problem, but not as exported interface). > All of those are still needed for the kernel. For #1 and #3, I agree if you limit the scope to driver model proper but I am not arguing kobject and all its friends should be abolished. I'm arguing that there is no reason to export it as API because it doesn't add any value exported. For #4, I don't know. This can be a matter of taste but I don't think #4 alone stands as justification for kobject as external API. For #2, I might be misundertanding. Please elaborate if I am. For uevent issue, I'll talk about more later. So, I honestly don't think the above four arguments successfully counter the original arguments. If I'm missing something, feel free to hammer me into enlightenment. :-) >> 4. So? >> >> From #3, as kobject no longer serves any valid purpose to sysfs, it's >> natural conclusion to try to remove kobject from sysfs, which of course >> brings up the question of conversion cost. > > I don't mind the removal of kobjects from sysfs in order to make sysfs > and kobjects work better/simpler. However the majority of the patches > you created to do this end up with more code overall, and are of no > benifit to the current users of sysfs and kobjects in the kernel. > >> 95+% of sysfs users use it through driver model which wraps sysfs >> interface and exports it as a part of driver model. For these, >> conversion only needs to happen inside the driver model, so we >> definitely can do that. > > But what would that benifit the driver model? There is no code reduction or functionality improvement yet because all of them are still using the compatibility interface. Properly converted, sysfs handling code all over the kernel can be _much_ simplified and more robust. I bet there are numerous bugs in sysfs creation failure handling path all over mid/low level drivers. New interface makes those bugs much less likely. >> The rest isn't great in number and, much more importantly, many of those >> suffer from the current interface which is painful to use independently. >> For example, kernel/module.c does all the kobject dances including >> defining a subsystem just to ignore everything else and use it as an >> opaque token to sysfs (kset_find_obj doesn't count, a generic map or >> sysfs with sysfs_dirent interface can do that just as well). > > I will not deny that the current use of kobjects/ksets/ktypes (subsystem > is now gone) is difficult and extreemly painful. I am currently working > to fix this issue. But don't think that the
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Thu, Sep 27, 2007 at 01:25:48PM -0600, Eric W. Biederman wrote: > > I still need to look at the code in detail but I have some concerns > I want to inject into this conversation of future sysfs architecture. > > - If we want to carefully limit sysfs from going to wild code review > is clearly not enough. We need some technological measures to > assist us. As the experience with sysctl has shown. I totally agree. You should see the ways that people have tried to circumvent the current kobject/sysfs code over the past years. It's so scary it's not even funny... > - The network namespace work scheduled to be merged in 2.6.24 is > currently has a dependency in Kconfig that is "&& !SYSFS" > because sysfs is currently very much a moving target. > > Does it look like we can resolve Tejun's work for 2.6.24? > If not does it make sense to push my patches that allow > multiple mounts of sysfs for 2.6.24? So I can allow > network namespaces in the presence of sysfs. > > Outside of sysfs and the device model I'm only talk maybe 30 lines > of code...So I could easily merge that patch later in the > merge window after the other pieces have gone in. I would be interested in seeing what your patches look like. I don't think that we should take any more sysfs changes for 2.6.24 as we do have a lot of them right now, and I don't think that Tejun and I agree on the future direction of the outstanding ones just yet. But I don't think that your multiple-mount patches could make it into .24, unless .23 is still weeks away. > - Farther down the road we have the device namespace. > The bounding requirements are: > - We want to restrict which set of devices a subset of process > can access. That's reasonable. > - When we migrate an application we want to preserve the device > numbers of all devices that show up in the new location. > So filesystems whose block devices reside on a SAN, ramdisks, > ttys, etc. > Other devices that really are different we can handle with > hotplug remove and add events, during the migration. > > So while there is lower hanging fruit the requirements for a > device namespace are becoming clear, and don't look like something > we will ultimately be able to dodge. > > For sysfs the implication is that we will need to filter the > hotplug events based upon the device namespace of the recipient, and > we will need to restrict the set of devices that show up in sysfs > based on who mounts it (as the prototype patches with the network > namespace are doing). That is going to be interesting to see how you come up with a way to do hat. > Also fun is that the dev file implementation needs to be able to > report different major:minor numbers based on which mount of > sysfs we are dealing with. Um, no, that's not going to happen. /dev/sda will _always_ have the same major:minor number, as defined by the LSB spec. You can not break that at all. So while you might not want to show all mounts /sys/devices/block/sda/ the ones that you do, will all have the LSB defined major:minor number assigned to it. thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Thu, Sep 27, 2007 at 04:35:07AM -0700, Tejun Heo wrote: > Hello, Greg. > > Sorry about the late reply. I'm sandwiched between several release > dates (I bet you know) and sudden burst of family/personal events (all > kinds of them - good, annual and bad). Same here, I'm swamped with stuff, and now am on vacation for a few days... > I suppose I failed to sell new sysfs_dirent based interface idea > face-to-face. I'll try it one more time on-line. I tend to do these > things better on-line, especially in English. So, please spare some > more time on the subject. > > IMHO, removal of kobject from sysfs interface is a logical and necessary > step toward easier driver model, not an unnecessary because-we-can > modification. I need to go back to what a "kobject" is to explain this. > > 1. What is a kobject? > > If I understood it correctly, kobject was separated out from device > driver model to allow entities outside of driver model to use sysfs, so > it's a part of device driver object which is necessary to interact with > sysfs. Yes, it was done so that we could have /sys/block for block devices. Al Viro did that work. > Originally, driver model objects and their sysfs representation was > tightly coupled. This is what made kobject a "kobject" not > "sysfs_something". Driver model and sysfs shared the same object to > represent kernel and sysfs-side. kobject was a base class of all driver > model objects, interaction with sysfs was through this base object and > implementation of sysfs also depended on kobject. > > The functionality served by kobject can be broken down into the > following two. > > F-a. To serve as an entity both subsystems can share lifespan > management. ie. both subsystems reference count on kobject. > > F-b. To serve as an entity both subsystems can base their internal > representation on. (base object in OO term). Yes, those are two functions, I can agree with. > 2. Implementation of immediate detach of sysfs nodes > > Unfortunately, this tight coupling caused several problems. One of the > most annoying problems was that userland was allowed to interfere > directly with lifespan management of kernel objects which formed basis > of driver model, causing quite some number of problems directly and > indirectly and unfortunately the problem couldn't be contained inside > driver model. Mid or low level driver implementation was affected too. > > As a response, immediate detach of sysfs nodes was implemented. When a > sysfs node is removed, it immediately disconnects from the associated > kobject. This way, the burden of lifespan management (at least sysfs > related part of it) is contained inside sysfs proper where we can afford > more effort, testing and thus complexity. On an unrelated note, I think > this is the beginning step toward a bigger change, that is, shielding > drivers from the complexity of object lifespan management. I agree. Your work in this area has been great and helped out a lot. > Anyways, so, now that immediate disconnect is in place, sysfs is no > longer involved in lifespan management of driver model objects. It > attaches and detaches when it's told to do so. Naturally, most of > internal implementation changed to use independent objects > (sysfs_dirent) instead of kobject in the process. > > 3. Where does that leave kobject? > > If you combine #1 and #2, both functionalities become questionable. > > F-a. sysfs no longer plays role in lifespan management of driver model > object. This functionality is exactly what's killed by #2. Yes, but a kobject is still needed internally for the lifespan management. > F-b. In the process of #2, the internal representation naturally moved > over to sysfs_dirent. The interface remained the same but after > dereferencing kobject->sd, kobject itself is mostly irrelevant to sysfs > and where kobj is still used, the code is either difficult to read or > outright buggy. This is expected. Lifespans of sysfs and driver model > objects are managed completely independently. Dereferencing objects on > the other domain is inherently cumbersome. > > With both F-a and F-b nullified, left purposes kobject still serve are > the followings. > > L-a. Serve as opaque token in sysfs interface but with all the reasons > to do so removed, this is at best cumbersome. It's an opaque token but > with a lot of unnecessary baggages. This role can be _much_ better > served by sysfs_dirent. Possibly, I'm still not sold on this. > L-b. Serve as something a subsystem can use to count references which > also can be used to access sysfs if wanted. To me, this feels like a a > flash light which can also be used to spread butter. Heh, that's a good analogy :) > IMHO, both L-a and L-b contribute only to obfuscation of the driver > model and sysfs. No. I think you are missing a number of things that kobjects provide and allow: - a structurual heirachy of devices. Combine kobjects with
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Thu, Sep 27, 2007 at 04:35:07AM -0700, Tejun Heo wrote: Hello, Greg. Sorry about the late reply. I'm sandwiched between several release dates (I bet you know) and sudden burst of family/personal events (all kinds of them - good, annual and bad). Same here, I'm swamped with stuff, and now am on vacation for a few days... I suppose I failed to sell new sysfs_dirent based interface idea face-to-face. I'll try it one more time on-line. I tend to do these things better on-line, especially in English. So, please spare some more time on the subject. IMHO, removal of kobject from sysfs interface is a logical and necessary step toward easier driver model, not an unnecessary because-we-can modification. I need to go back to what a kobject is to explain this. 1. What is a kobject? If I understood it correctly, kobject was separated out from device driver model to allow entities outside of driver model to use sysfs, so it's a part of device driver object which is necessary to interact with sysfs. Yes, it was done so that we could have /sys/block for block devices. Al Viro did that work. Originally, driver model objects and their sysfs representation was tightly coupled. This is what made kobject a kobject not sysfs_something. Driver model and sysfs shared the same object to represent kernel and sysfs-side. kobject was a base class of all driver model objects, interaction with sysfs was through this base object and implementation of sysfs also depended on kobject. The functionality served by kobject can be broken down into the following two. F-a. To serve as an entity both subsystems can share lifespan management. ie. both subsystems reference count on kobject. F-b. To serve as an entity both subsystems can base their internal representation on. (base object in OO term). Yes, those are two functions, I can agree with. 2. Implementation of immediate detach of sysfs nodes Unfortunately, this tight coupling caused several problems. One of the most annoying problems was that userland was allowed to interfere directly with lifespan management of kernel objects which formed basis of driver model, causing quite some number of problems directly and indirectly and unfortunately the problem couldn't be contained inside driver model. Mid or low level driver implementation was affected too. As a response, immediate detach of sysfs nodes was implemented. When a sysfs node is removed, it immediately disconnects from the associated kobject. This way, the burden of lifespan management (at least sysfs related part of it) is contained inside sysfs proper where we can afford more effort, testing and thus complexity. On an unrelated note, I think this is the beginning step toward a bigger change, that is, shielding drivers from the complexity of object lifespan management. I agree. Your work in this area has been great and helped out a lot. Anyways, so, now that immediate disconnect is in place, sysfs is no longer involved in lifespan management of driver model objects. It attaches and detaches when it's told to do so. Naturally, most of internal implementation changed to use independent objects (sysfs_dirent) instead of kobject in the process. 3. Where does that leave kobject? If you combine #1 and #2, both functionalities become questionable. F-a. sysfs no longer plays role in lifespan management of driver model object. This functionality is exactly what's killed by #2. Yes, but a kobject is still needed internally for the lifespan management. F-b. In the process of #2, the internal representation naturally moved over to sysfs_dirent. The interface remained the same but after dereferencing kobject-sd, kobject itself is mostly irrelevant to sysfs and where kobj is still used, the code is either difficult to read or outright buggy. This is expected. Lifespans of sysfs and driver model objects are managed completely independently. Dereferencing objects on the other domain is inherently cumbersome. With both F-a and F-b nullified, left purposes kobject still serve are the followings. L-a. Serve as opaque token in sysfs interface but with all the reasons to do so removed, this is at best cumbersome. It's an opaque token but with a lot of unnecessary baggages. This role can be _much_ better served by sysfs_dirent. Possibly, I'm still not sold on this. L-b. Serve as something a subsystem can use to count references which also can be used to access sysfs if wanted. To me, this feels like a a flash light which can also be used to spread butter. Heh, that's a good analogy :) IMHO, both L-a and L-b contribute only to obfuscation of the driver model and sysfs. No. I think you are missing a number of things that kobjects provide and allow: - a structurual heirachy of devices. Combine kobjects with ksets and ktypes, and you have a very powerful system to categorize objects and
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Thu, Sep 27, 2007 at 01:25:48PM -0600, Eric W. Biederman wrote: I still need to look at the code in detail but I have some concerns I want to inject into this conversation of future sysfs architecture. - If we want to carefully limit sysfs from going to wild code review is clearly not enough. We need some technological measures to assist us. As the experience with sysctl has shown. I totally agree. You should see the ways that people have tried to circumvent the current kobject/sysfs code over the past years. It's so scary it's not even funny... - The network namespace work scheduled to be merged in 2.6.24 is currently has a dependency in Kconfig that is !SYSFS because sysfs is currently very much a moving target. Does it look like we can resolve Tejun's work for 2.6.24? If not does it make sense to push my patches that allow multiple mounts of sysfs for 2.6.24? So I can allow network namespaces in the presence of sysfs. Outside of sysfs and the device model I'm only talk maybe 30 lines of code...So I could easily merge that patch later in the merge window after the other pieces have gone in. I would be interested in seeing what your patches look like. I don't think that we should take any more sysfs changes for 2.6.24 as we do have a lot of them right now, and I don't think that Tejun and I agree on the future direction of the outstanding ones just yet. But I don't think that your multiple-mount patches could make it into .24, unless .23 is still weeks away. - Farther down the road we have the device namespace. The bounding requirements are: - We want to restrict which set of devices a subset of process can access. That's reasonable. - When we migrate an application we want to preserve the device numbers of all devices that show up in the new location. So filesystems whose block devices reside on a SAN, ramdisks, ttys, etc. Other devices that really are different we can handle with hotplug remove and add events, during the migration. So while there is lower hanging fruit the requirements for a device namespace are becoming clear, and don't look like something we will ultimately be able to dodge. For sysfs the implication is that we will need to filter the hotplug events based upon the device namespace of the recipient, and we will need to restrict the set of devices that show up in sysfs based on who mounts it (as the prototype patches with the network namespace are doing). That is going to be interesting to see how you come up with a way to do hat. Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Um, no, that's not going to happen. /dev/sda will _always_ have the same major:minor number, as defined by the LSB spec. You can not break that at all. So while you might not want to show all mounts /sys/devices/block/sda/ the ones that you do, will all have the LSB defined major:minor number assigned to it. thanks, greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Hello, Greg. I think this definitely needs more discussion, so here we go... Greg KH wrote: 1. What is a kobject? [--snip--] The functionality served by kobject can be broken down into the following two. F-a. To serve as an entity both subsystems can share lifespan management. ie. both subsystems reference count on kobject. F-b. To serve as an entity both subsystems can base their internal representation on. (base object in OO term). Yes, those are two functions, I can agree with. [--snip--] 3. Where does that leave kobject? If you combine #1 and #2, both functionalities become questionable. F-a. sysfs no longer plays role in lifespan management of driver model object. This functionality is exactly what's killed by #2. Yes, but a kobject is still needed internally for the lifespan management. Yes, exactly - internally to the driver model (or drivers which ride along). To sysfs, it has no function other than being an opaque token. [--snip--] IMHO, both L-a and L-b contribute only to obfuscation of the driver model and sysfs. No. I think you are missing a number of things that kobjects provide and allow: - a structurual heirachy of devices. Combine kobjects with ksets and ktypes, and you have a very powerful system to categorize objects and their representation to each other. Yes, which only needs to be used _inside_ the driver model implementation proper. - a consistant and easy interface to userspace through uevents/hotplug of the creation and removal of these objects. This keeps the different parts of the kernel that need this interface from having to create it every time on their own. Things can be much easier than now. Also, the above paragraph is inconsistent with the rest of your argument or am I misunderstanding what you mean by the above paragraph? - a way to easily create and export attributes in sysfs automatically. This is and should be the function of the driver model not kobjects. Removing kobject from the interface doesn't change anything about this. - a way to provide working reference counting for a variety of different objects. To me, this just feels wrong and does more harm than it helps. I really think we shouldn't have multi-role flash light at the core of our driver model (inside driver model proper, no problem, but not as exported interface). All of those are still needed for the kernel. For #1 and #3, I agree if you limit the scope to driver model proper but I am not arguing kobject and all its friends should be abolished. I'm arguing that there is no reason to export it as API because it doesn't add any value exported. For #4, I don't know. This can be a matter of taste but I don't think #4 alone stands as justification for kobject as external API. For #2, I might be misundertanding. Please elaborate if I am. For uevent issue, I'll talk about more later. So, I honestly don't think the above four arguments successfully counter the original arguments. If I'm missing something, feel free to hammer me into enlightenment. :-) 4. So? From #3, as kobject no longer serves any valid purpose to sysfs, it's natural conclusion to try to remove kobject from sysfs, which of course brings up the question of conversion cost. I don't mind the removal of kobjects from sysfs in order to make sysfs and kobjects work better/simpler. However the majority of the patches you created to do this end up with more code overall, and are of no benifit to the current users of sysfs and kobjects in the kernel. 95+% of sysfs users use it through driver model which wraps sysfs interface and exports it as a part of driver model. For these, conversion only needs to happen inside the driver model, so we definitely can do that. But what would that benifit the driver model? There is no code reduction or functionality improvement yet because all of them are still using the compatibility interface. Properly converted, sysfs handling code all over the kernel can be _much_ simplified and more robust. I bet there are numerous bugs in sysfs creation failure handling path all over mid/low level drivers. New interface makes those bugs much less likely. The rest isn't great in number and, much more importantly, many of those suffer from the current interface which is painful to use independently. For example, kernel/module.c does all the kobject dances including defining a subsystem just to ignore everything else and use it as an opaque token to sysfs (kset_find_obj doesn't count, a generic map or sysfs with sysfs_dirent interface can do that just as well). I will not deny that the current use of kobjects/ksets/ktypes (subsystem is now gone) is difficult and extreemly painful. I am currently working to fix this issue. But don't think that the reason this is hard to use means that it should be abolished alltogether. Rather, it means
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH [EMAIL PROTECTED] writes: Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Um, no, that's not going to happen. /dev/sda will _always_ have the same major:minor number, as defined by the LSB spec. You can not break that at all. So while you might not want to show all mounts /sys/devices/block/sda/ the ones that you do, will all have the LSB defined major:minor number assigned to it. Hmm. If that is in the LSB it must come from Documentation/devices.txt I'm not after changing the user visible major/minor assignments. Let me see if a concrete example will help. Suppose I have have a SAN with two disks: disk-1 and disk-2. I have two machines A and B. On machine A I get the mapping: sda - disk-1, sdb -disk-2. On machine B I wind up with a different probe order so I get the mapping: sda - disk-2 sdb -disk-1. To be very clear by sda I mean the block device with major 8 and minor 0, and by sdb I mean the block device with major 8 and minor 16. So I decide I want an environment on machine B that looks just like the environment on machine A, so I can bring transfer over a running program or whatever. So I run around looking at UUID labels and what not and I discover that the machine B knows disk-1 as sdb and that machine A knows disk-1 as sda. So I want to say: /sys/devices/block/sdb show up in this other device namespace as /sys/devices/block/sda. In that instance a running program won't notice the difference. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Greg KH [EMAIL PROTECTED] writes:
I would be interested in seeing what your patches look like.
Sure.
I don't
think that we should take any more sysfs changes for 2.6.24 as we do
have a lot of them right now, and I don't think that Tejun and I agree
on the future direction of the outstanding ones just yet.
Sounds reasonable.
But I don't think that your multiple-mount patches could make it into
.24, unless .23 is still weeks away.
Well I have posted them all earlier. At this point I it makes most
sense to wait until after the big merge happen and every rebases on
top of that. Then everyone will have network namespace support and
it is easier to look through all of the patches. Especially since
it looks like the merge window will open any day now.
I will quickly recap the essence of what I am looking at:
On directories of interest I tag all of their directory
entries with which namespace they belong to. On a mount
of sysfs I remember which namespace we were in when we
mounted sysfs. The I filter readdir and lookup based upon
the namespace I captured at mount time. I do my best
to generalize it so that the logic can work for different
namespaces.
Currently the heart of the patch from the network namespace
is below (I sniped the part that does the capture at mount time).
Basically the interface to users of this functionality is just
providing some way to go from a super block or a kobject to
the tag sysfs is using to filter things.
So I get one sysfs_dirent tree, but each super_block has it's
own tree of dcache entries.
Everything else is pretty much details in checking and propagating
the tags into the appropriate places.
Eric
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 5adfdc2..a300f6e 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -435,6 +437,23 @@ static void netdev_release(struct device *d)
kfree((char *)dev - dev-padded);
}
+static const void *net_sb_tag(struct sysfs_tag_info *info)
+{
+ return info-net_ns;
+}
+
+static const void *net_kobject_tag(struct kobject *kobj)
+{
+ struct net_device *dev;
+ dev = container_of(kobj, struct net_device, dev.kobj);
+ return dev-nd_net;
+}
+
+static const struct sysfs_tagged_dir_operations net_tagged_dir_operations = {
+ .sb_tag = net_sb_tag,
+ .kobject_tag = net_kobject_tag,
+};
+
static struct class net_class = {
.name = net,
.dev_release = netdev_release,
@@ -444,6 +463,7 @@ static struct class net_class = {
#ifdef CONFIG_HOTPLUG
.dev_uevent = netdev_uevent,
#endif
+ .tag_ops = net_tagged_dir_operations,
};
/* Delete sysfs entries but hold kobject reference until after all
--
1.5.3.rc6.17.g1911
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Devel] Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Eric W. Biederman wrote: Greg KH [EMAIL PROTECTED] writes: Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Um, no, that's not going to happen. /dev/sda will _always_ have the same major:minor number, as defined by the LSB spec. You can not break that at all. So while you might not want to show all mounts /sys/devices/block/sda/ the ones that you do, will all have the LSB defined major:minor number assigned to it. Hmm. If that is in the LSB it must come from Documentation/devices.txt I'm not after changing the user visible major/minor assignments. Let me see if a concrete example will help. Suppose I have have a SAN with two disks: disk-1 and disk-2. I have two machines A and B. On machine A I get the mapping: sda - disk-1, sdb -disk-2. On machine B I wind up with a different probe order so I get the mapping: sda - disk-2 sdb -disk-1. To be very clear by sda I mean the block device with major 8 and minor 0, and by sdb I mean the block device with major 8 and minor 16. So I decide I want an environment on machine B that looks just like the environment on machine A, so I can bring transfer over a running program or whatever. So I run around looking at UUID labels and what not and I discover that the machine B knows disk-1 as sdb and that machine A knows disk-1 as sda. So I want to say: /sys/devices/block/sdb show up in this other device namespace as /sys/devices/block/sda. Imho environments to be migratable should have no direct access to the devices. You can use any of stacked virtual filesystems to hide real device from container. You will have problems much bigger than this one otherwise (imagine access to video, sound etc.) Kirill - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Devel] Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Kirill Korotaev [EMAIL PROTECTED] writes: Imho environments to be migratable should have no direct access to the devices. You can use any of stacked virtual filesystems to hide real device from container. You will have problems much bigger than this one otherwise (imagine access to video, sound etc.) What I am primarily concern about is when you can make the case that the hardware we are talking is present before and after the migration. When you are directly accessing a device. For times when it makes sense to directly access hardware in a container (think infiniband OS-bypass NICs). We need to tell user space that the device was unplugged and another one was plugged in. If user space can cope with that things should continue to work. There are some very specific cases that we can support: - Stateless devices like /dev/zero and dev/random. - Virtual devices like ttys, ramdisks, loop devices - Remote block devices like SCSI disks on a san, iSCSI, nbd, ATAoE. - Local pseudo block devices like the backing devices for virtual filesystems. There are very specific limits in which this can work and be useable, and I don't claim to have looked at all of the details, but for the block device case in particular we export the block device number to user space in stat. There are some common applications which do memorize stat data for files things like: git, incremental backup software, and intrusion detection software. Frankly the times when this matters is rare enough I don't put a big priority on getting this done quickly. However a strong case has been made for all of this filtering so we can run things like udev in a container. Initially I only expect stateless character devices and ttys to be allowed in a namespace, and I don't have a clue what device number we will use in st_dev for stat in the case our block device isn't in the user space interface. I just know that it sounds like where we want to be eventually and thinking about it now isn't a problem. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Hello, Eric. Eric W. Biederman wrote: > Mostly I am thinking that any non-object model users should have > their own dedicated wrapper layer. To help keep things consistent > and to make it hard enough to abuse the system that people will > find that it is usually easier to do it the right way. Hmmm... I think most current non-driver-model sysfs users are deep in kernel anyway, but I think not exporting sysfs interface at all might be a bit too restrictive. I think we need to examine the current non-driver-model sysfs users thoroughly to determine what to do about this. But, yes, I do agree that we need to put restrictions one way or the other. > Does it look like we can resolve Tejun's work for 2.6.24? > If not does it make sense to push my patches that allow > multiple mounts of sysfs for 2.6.24? So I can allow > network namespaces in the presence of sysfs. > > Outside of sysfs and the device model I'm only talk maybe 30 lines > of code...So I could easily merge that patch later in the > merge window after the other pieces have gone in. I think it would be better if namespace comes after interface update and other new features, especially symlink renaming, but, under the current circumstance, it might delay namespace unnecessarily and I have no problem with your patches going in first. My concerns are... * Do you think you can use new rename implementation contained in this series? It borrows basic ideas from the implementation you used for namespace but is more generic. It would be great if you can use it without too much modification. * I'm still against using callbacks to determine namespace tags because callbacks need to be coupled with sysfs internals more tightly and are more difficult to grasp interface-wise. > - Farther down the road we have the device namespace. > The bounding requirements are: > - We want to restrict which set of devices a subset of process > can access. > - When we migrate an application we want to preserve the device > numbers of all devices that show up in the new location. > So filesystems whose block devices reside on a SAN, ramdisks, > ttys, etc. > Other devices that really are different we can handle with > hotplug remove and add events, during the migration. > > So while there is lower hanging fruit the requirements for a > device namespace are becoming clear, and don't look like something > we will ultimately be able to dodge. > > For sysfs the implication is that we will need to filter the > hotplug events based upon the device namespace of the recipient, and > we will need to restrict the set of devices that show up in sysfs > based on who mounts it (as the prototype patches with the network > namespace are doing). > > Also fun is that the dev file implementation needs to be able to > report different major:minor numbers based on which mount of > sysfs we are dealing with. Ah... Coming few months will be fun, won't they? :-) -- tejun - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Hello, Eric. Eric W. Biederman wrote: Mostly I am thinking that any non-object model users should have their own dedicated wrapper layer. To help keep things consistent and to make it hard enough to abuse the system that people will find that it is usually easier to do it the right way. Hmmm... I think most current non-driver-model sysfs users are deep in kernel anyway, but I think not exporting sysfs interface at all might be a bit too restrictive. I think we need to examine the current non-driver-model sysfs users thoroughly to determine what to do about this. But, yes, I do agree that we need to put restrictions one way or the other. Does it look like we can resolve Tejun's work for 2.6.24? If not does it make sense to push my patches that allow multiple mounts of sysfs for 2.6.24? So I can allow network namespaces in the presence of sysfs. Outside of sysfs and the device model I'm only talk maybe 30 lines of code...So I could easily merge that patch later in the merge window after the other pieces have gone in. I think it would be better if namespace comes after interface update and other new features, especially symlink renaming, but, under the current circumstance, it might delay namespace unnecessarily and I have no problem with your patches going in first. My concerns are... * Do you think you can use new rename implementation contained in this series? It borrows basic ideas from the implementation you used for namespace but is more generic. It would be great if you can use it without too much modification. * I'm still against using callbacks to determine namespace tags because callbacks need to be coupled with sysfs internals more tightly and are more difficult to grasp interface-wise. - Farther down the road we have the device namespace. The bounding requirements are: - We want to restrict which set of devices a subset of process can access. - When we migrate an application we want to preserve the device numbers of all devices that show up in the new location. So filesystems whose block devices reside on a SAN, ramdisks, ttys, etc. Other devices that really are different we can handle with hotplug remove and add events, during the migration. So while there is lower hanging fruit the requirements for a device namespace are becoming clear, and don't look like something we will ultimately be able to dodge. For sysfs the implication is that we will need to filter the hotplug events based upon the device namespace of the recipient, and we will need to restrict the set of devices that show up in sysfs based on who mounts it (as the prototype patches with the network namespace are doing). Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Ah... Coming few months will be fun, won't they? :-) -- tejun - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
I still need to look at the code in detail but I have some concerns I want to inject into this conversation of future sysfs architecture. - If we want to carefully limit sysfs from going to wild code review is clearly not enough. We need some technological measures to assist us. As the experience with sysctl has shown. I discovered that something like 10% of the sysctl entries were buggy and had been for years when I added basic runtime sanity checks. I had also found one instance in the kernel and had one instance from outside the kernel where people had created files under /proc/sys not as sysctls but as using the infrastructure from proc_generic.c because it happened to work. So while it very well may be we don't want to use the kobject interface anymore. I expect that we want to have the sysfs_dirent interface not exported to modules, and only allow direct access from code compiled into the kernel. Mostly I am thinking that any non-object model users should have their own dedicated wrapper layer. To help keep things consistent and to make it hard enough to abuse the system that people will find that it is usually easier to do it the right way. - The network namespace work scheduled to be merged in 2.6.24 is currently has a dependency in Kconfig that is "&& !SYSFS" because sysfs is currently very much a moving target. Does it look like we can resolve Tejun's work for 2.6.24? If not does it make sense to push my patches that allow multiple mounts of sysfs for 2.6.24? So I can allow network namespaces in the presence of sysfs. Outside of sysfs and the device model I'm only talk maybe 30 lines of code...So I could easily merge that patch later in the merge window after the other pieces have gone in. - Farther down the road we have the device namespace. The bounding requirements are: - We want to restrict which set of devices a subset of process can access. - When we migrate an application we want to preserve the device numbers of all devices that show up in the new location. So filesystems whose block devices reside on a SAN, ramdisks, ttys, etc. Other devices that really are different we can handle with hotplug remove and add events, during the migration. So while there is lower hanging fruit the requirements for a device namespace are becoming clear, and don't look like something we will ultimately be able to dodge. For sysfs the implication is that we will need to filter the hotplug events based upon the device namespace of the recipient, and we will need to restrict the set of devices that show up in sysfs based on who mounts it (as the prototype patches with the network namespace are doing). Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Hello, Greg. Sorry about the late reply. I'm sandwiched between several release dates (I bet you know) and sudden burst of family/personal events (all kinds of them - good, annual and bad). Greg KH wrote: >> * sysfs becomes a separate module and driver model becomes a user of >> sysfs. Those two are not entangled anymore. Things are easier to >> understand and test this way. > > This is good, I like this. Great. :-) >> * Non-driver model users of sysfs (modules, blkdev, etc...) don't have >> to jump through hoops to use sysfs. kobj based interface requires >> attribute wrapping and is awkward to use directly. Also, the user >> is required to create a dummy kobj which doesn't serve much purpose >> than being a token for sysfs reference. New sysfs-dirent based >> interface is straight forward proc-fs like interface and should be >> easier and more intuitive for those users. > > This is not good, I don't like this :( > > As we spoke a few weeks ago, the non-driver model users of sysfs are ok. > Yes, it's not trivial to use sysfs in this manner, and it should be made > easier, but we still need to keep our tree of objects. Using a kobject > for sysfs access is a good thing as it provides a tiny grasp on keeping > the usage of sysfs under control. > > So while I like the separation of sysfs and kobjects from an > architectural and conceptual level for testing and understanding, I do > not want to allow the use of sysfs without creating a backing kobject > like we do today. > > I'm all for making the "raw" kobject access easier, cleaning up the > attribute "mess" that you need to go through. The cleanups that Kay and > I have been doing in the kset and subsystem area are steps in that > direction and I have more I want to do there to help make it easier to > use and understand. I suppose I failed to sell new sysfs_dirent based interface idea face-to-face. I'll try it one more time on-line. I tend to do these things better on-line, especially in English. So, please spare some more time on the subject. IMHO, removal of kobject from sysfs interface is a logical and necessary step toward easier driver model, not an unnecessary because-we-can modification. I need to go back to what a "kobject" is to explain this. 1. What is a kobject? If I understood it correctly, kobject was separated out from device driver model to allow entities outside of driver model to use sysfs, so it's a part of device driver object which is necessary to interact with sysfs. Originally, driver model objects and their sysfs representation was tightly coupled. This is what made kobject a "kobject" not "sysfs_something". Driver model and sysfs shared the same object to represent kernel and sysfs-side. kobject was a base class of all driver model objects, interaction with sysfs was through this base object and implementation of sysfs also depended on kobject. The functionality served by kobject can be broken down into the following two. F-a. To serve as an entity both subsystems can share lifespan management. ie. both subsystems reference count on kobject. F-b. To serve as an entity both subsystems can base their internal representation on. (base object in OO term). 2. Implementation of immediate detach of sysfs nodes Unfortunately, this tight coupling caused several problems. One of the most annoying problems was that userland was allowed to interfere directly with lifespan management of kernel objects which formed basis of driver model, causing quite some number of problems directly and indirectly and unfortunately the problem couldn't be contained inside driver model. Mid or low level driver implementation was affected too. As a response, immediate detach of sysfs nodes was implemented. When a sysfs node is removed, it immediately disconnects from the associated kobject. This way, the burden of lifespan management (at least sysfs related part of it) is contained inside sysfs proper where we can afford more effort, testing and thus complexity. On an unrelated note, I think this is the beginning step toward a bigger change, that is, shielding drivers from the complexity of object lifespan management. Anyways, so, now that immediate disconnect is in place, sysfs is no longer involved in lifespan management of driver model objects. It attaches and detaches when it's told to do so. Naturally, most of internal implementation changed to use independent objects (sysfs_dirent) instead of kobject in the process. 3. Where does that leave kobject? If you combine #1 and #2, both functionalities become questionable. F-a. sysfs no longer plays role in lifespan management of driver model object. This functionality is exactly what's killed by #2. F-b. In the process of #2, the internal representation naturally moved over to sysfs_dirent. The interface remained the same but after dereferencing kobject->sd, kobject itself is mostly irrelevant to sysfs and where kobj is still used, the code is
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
Hello, Greg. Sorry about the late reply. I'm sandwiched between several release dates (I bet you know) and sudden burst of family/personal events (all kinds of them - good, annual and bad). Greg KH wrote: * sysfs becomes a separate module and driver model becomes a user of sysfs. Those two are not entangled anymore. Things are easier to understand and test this way. This is good, I like this. Great. :-) * Non-driver model users of sysfs (modules, blkdev, etc...) don't have to jump through hoops to use sysfs. kobj based interface requires attribute wrapping and is awkward to use directly. Also, the user is required to create a dummy kobj which doesn't serve much purpose than being a token for sysfs reference. New sysfs-dirent based interface is straight forward proc-fs like interface and should be easier and more intuitive for those users. This is not good, I don't like this :( As we spoke a few weeks ago, the non-driver model users of sysfs are ok. Yes, it's not trivial to use sysfs in this manner, and it should be made easier, but we still need to keep our tree of objects. Using a kobject for sysfs access is a good thing as it provides a tiny grasp on keeping the usage of sysfs under control. So while I like the separation of sysfs and kobjects from an architectural and conceptual level for testing and understanding, I do not want to allow the use of sysfs without creating a backing kobject like we do today. I'm all for making the raw kobject access easier, cleaning up the attribute mess that you need to go through. The cleanups that Kay and I have been doing in the kset and subsystem area are steps in that direction and I have more I want to do there to help make it easier to use and understand. I suppose I failed to sell new sysfs_dirent based interface idea face-to-face. I'll try it one more time on-line. I tend to do these things better on-line, especially in English. So, please spare some more time on the subject. IMHO, removal of kobject from sysfs interface is a logical and necessary step toward easier driver model, not an unnecessary because-we-can modification. I need to go back to what a kobject is to explain this. 1. What is a kobject? If I understood it correctly, kobject was separated out from device driver model to allow entities outside of driver model to use sysfs, so it's a part of device driver object which is necessary to interact with sysfs. Originally, driver model objects and their sysfs representation was tightly coupled. This is what made kobject a kobject not sysfs_something. Driver model and sysfs shared the same object to represent kernel and sysfs-side. kobject was a base class of all driver model objects, interaction with sysfs was through this base object and implementation of sysfs also depended on kobject. The functionality served by kobject can be broken down into the following two. F-a. To serve as an entity both subsystems can share lifespan management. ie. both subsystems reference count on kobject. F-b. To serve as an entity both subsystems can base their internal representation on. (base object in OO term). 2. Implementation of immediate detach of sysfs nodes Unfortunately, this tight coupling caused several problems. One of the most annoying problems was that userland was allowed to interfere directly with lifespan management of kernel objects which formed basis of driver model, causing quite some number of problems directly and indirectly and unfortunately the problem couldn't be contained inside driver model. Mid or low level driver implementation was affected too. As a response, immediate detach of sysfs nodes was implemented. When a sysfs node is removed, it immediately disconnects from the associated kobject. This way, the burden of lifespan management (at least sysfs related part of it) is contained inside sysfs proper where we can afford more effort, testing and thus complexity. On an unrelated note, I think this is the beginning step toward a bigger change, that is, shielding drivers from the complexity of object lifespan management. Anyways, so, now that immediate disconnect is in place, sysfs is no longer involved in lifespan management of driver model objects. It attaches and detaches when it's told to do so. Naturally, most of internal implementation changed to use independent objects (sysfs_dirent) instead of kobject in the process. 3. Where does that leave kobject? If you combine #1 and #2, both functionalities become questionable. F-a. sysfs no longer plays role in lifespan management of driver model object. This functionality is exactly what's killed by #2. F-b. In the process of #2, the internal representation naturally moved over to sysfs_dirent. The interface remained the same but after dereferencing kobject-sd, kobject itself is mostly irrelevant to sysfs and where kobj is still used, the code is either difficult to read or outright buggy. This is
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
I still need to look at the code in detail but I have some concerns I want to inject into this conversation of future sysfs architecture. - If we want to carefully limit sysfs from going to wild code review is clearly not enough. We need some technological measures to assist us. As the experience with sysctl has shown. I discovered that something like 10% of the sysctl entries were buggy and had been for years when I added basic runtime sanity checks. I had also found one instance in the kernel and had one instance from outside the kernel where people had created files under /proc/sys not as sysctls but as using the infrastructure from proc_generic.c because it happened to work. So while it very well may be we don't want to use the kobject interface anymore. I expect that we want to have the sysfs_dirent interface not exported to modules, and only allow direct access from code compiled into the kernel. Mostly I am thinking that any non-object model users should have their own dedicated wrapper layer. To help keep things consistent and to make it hard enough to abuse the system that people will find that it is usually easier to do it the right way. - The network namespace work scheduled to be merged in 2.6.24 is currently has a dependency in Kconfig that is !SYSFS because sysfs is currently very much a moving target. Does it look like we can resolve Tejun's work for 2.6.24? If not does it make sense to push my patches that allow multiple mounts of sysfs for 2.6.24? So I can allow network namespaces in the presence of sysfs. Outside of sysfs and the device model I'm only talk maybe 30 lines of code...So I could easily merge that patch later in the merge window after the other pieces have gone in. - Farther down the road we have the device namespace. The bounding requirements are: - We want to restrict which set of devices a subset of process can access. - When we migrate an application we want to preserve the device numbers of all devices that show up in the new location. So filesystems whose block devices reside on a SAN, ramdisks, ttys, etc. Other devices that really are different we can handle with hotplug remove and add events, during the migration. So while there is lower hanging fruit the requirements for a device namespace are becoming clear, and don't look like something we will ultimately be able to dodge. For sysfs the implication is that we will need to filter the hotplug events based upon the device namespace of the recipient, and we will need to restrict the set of devices that show up in sysfs based on who mounts it (as the prototype patches with the network namespace are doing). Also fun is that the dev file implementation needs to be able to report different major:minor numbers based on which mount of sysfs we are dealing with. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Thu, Sep 20, 2007 at 05:05:39PM +0900, Tejun Heo wrote: > Subject: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model > > Hello, all. > > This is the third patchset of four sysfs update patchset series[1] and > to be applied on top of the second patchset[2]. > > Currently, sysfs interface is based on kobj. This made more sense > before because lifetime of sysfs nodes were tracked using kobj > reference counts. However, this is no longer true. sysfs nodes are > represented with a sysfs_dirent and external reference is severed > immediately on node removal. The internal implementation reflects > that too and mostly handles sysfs_dirents. > > This patchset divorces sysfs from kobject and driver model by > implementing sysfs_dirent based interface. This has the following > advantages. > > * sysfs becomes a separate module and driver model becomes a user of > sysfs. Those two are not entangled anymore. Things are easier to > understand and test this way. This is good, I like this. > * Non-driver model users of sysfs (modules, blkdev, etc...) don't have > to jump through hoops to use sysfs. kobj based interface requires > attribute wrapping and is awkward to use directly. Also, the user > is required to create a dummy kobj which doesn't serve much purpose > than being a token for sysfs reference. New sysfs-dirent based > interface is straight forward proc-fs like interface and should be > easier and more intuitive for those users. This is not good, I don't like this :( As we spoke a few weeks ago, the non-driver model users of sysfs are ok. Yes, it's not trivial to use sysfs in this manner, and it should be made easier, but we still need to keep our tree of objects. Using a kobject for sysfs access is a good thing as it provides a tiny grasp on keeping the usage of sysfs under control. So while I like the separation of sysfs and kobjects from an architectural and conceptual level for testing and understanding, I do not want to allow the use of sysfs without creating a backing kobject like we do today. I'm all for making the "raw" kobject access easier, cleaning up the attribute "mess" that you need to go through. The cleanups that Kay and I have been doing in the kset and subsystem area are steps in that direction and I have more I want to do there to help make it easier to use and understand. So, I'll try to pick and choose from this patchset what I feel is ok for now. Or does it depend on the second set of patches that are yet to be applied due to disagreements about module lifetimes? thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model
On Thu, Sep 20, 2007 at 05:05:39PM +0900, Tejun Heo wrote: Subject: [PATCHSET 3/4] sysfs: divorce sysfs from kobject and driver model Hello, all. This is the third patchset of four sysfs update patchset series[1] and to be applied on top of the second patchset[2]. Currently, sysfs interface is based on kobj. This made more sense before because lifetime of sysfs nodes were tracked using kobj reference counts. However, this is no longer true. sysfs nodes are represented with a sysfs_dirent and external reference is severed immediately on node removal. The internal implementation reflects that too and mostly handles sysfs_dirents. This patchset divorces sysfs from kobject and driver model by implementing sysfs_dirent based interface. This has the following advantages. * sysfs becomes a separate module and driver model becomes a user of sysfs. Those two are not entangled anymore. Things are easier to understand and test this way. This is good, I like this. * Non-driver model users of sysfs (modules, blkdev, etc...) don't have to jump through hoops to use sysfs. kobj based interface requires attribute wrapping and is awkward to use directly. Also, the user is required to create a dummy kobj which doesn't serve much purpose than being a token for sysfs reference. New sysfs-dirent based interface is straight forward proc-fs like interface and should be easier and more intuitive for those users. This is not good, I don't like this :( As we spoke a few weeks ago, the non-driver model users of sysfs are ok. Yes, it's not trivial to use sysfs in this manner, and it should be made easier, but we still need to keep our tree of objects. Using a kobject for sysfs access is a good thing as it provides a tiny grasp on keeping the usage of sysfs under control. So while I like the separation of sysfs and kobjects from an architectural and conceptual level for testing and understanding, I do not want to allow the use of sysfs without creating a backing kobject like we do today. I'm all for making the raw kobject access easier, cleaning up the attribute mess that you need to go through. The cleanups that Kay and I have been doing in the kset and subsystem area are steps in that direction and I have more I want to do there to help make it easier to use and understand. So, I'll try to pick and choose from this patchset what I feel is ok for now. Or does it depend on the second set of patches that are yet to be applied due to disagreements about module lifetimes? thanks, greg k-h - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
