Re: debian and UDEV
Am Freitag, 19. Mai 2006 03:14 schrieb Goswin von Brederlow: Hendrik Sattler [EMAIL PROTECTED] writes: Am Donnerstag, 18. Mai 2006 21:53 schrieb Goswin von Brederlow: But how do you detect when there is no device to be found to call the function? That's of absolutely no interest because in this case, you cannot go on, having synchronous events or not. However, you might have alternatives for a root partition that are fulfilled with another function call. The point is that to keep the goal in mind: you want to achieve something e.g. by probing many different possibilities. Lets say you want to find a scanner: you probe SCSI busses, spawn a thread that test the parallel bus and another one for USB and whatever else. You only want to find one. You don't synchronize on any of those specific events but on the result: that a scanner is found. Sure, you need a timeout, but it is unrelated to a specific probing method So you mean you register a function for block device found and then scan all the ide, sata, scsi, usb, ... drivers from the initrd and for every disk and partition the function gets called. When it gets called with the partition with the right signature (label or uuid for example) it mounts / and continues. Not a function for scsi disk ID 0 on scsi0 (3ware) found. I see what you mean. Keeping the function abstract enough allows removing a disk and plug it in somewhere else, e.g. change the scsi ID. Almost. It can still be scsi disk ID 0 on scsi0 (3ware) found but that it a matter of only the script to mount root and /etc/fstab (which also allows for LABEL and UUID, as alternative to the links in /dev created by udev), not the other script that wants root mounted and waits until this is the case. However, I am not sure that this is really done this way, currently. Would be the cleanest approach, though. HS pgpuHwhMZQpCq.pgp Description: PGP signature
Re: debian and UDEV
Hello, On Tue, 16.05.2006 at 04:05:41 +, Brian M. Carlson [EMAIL PROTECTED] wrote: [ udev being, umm, not very nice ] Need I say more? yes: Please say *why* newer 2.6.x kernels actually do depend on udev instead of hotplug. Thank you! Best, --Toni++ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Am Donnerstag 18 Mai 2006 11:25 schrieb Toni Mueller: Hello, On Tue, 16.05.2006 at 04:05:41 +, Brian M. Carlson [EMAIL PROTECTED] wrote: [ udev being, umm, not very nice ] Need I say more? yes: Please say *why* newer 2.6.x kernels actually do depend on udev instead of hotplug. Thank you! Best, --Toni++ Since hotplug's deprecation, udev has taken on the role of coldplugger, hotplugger. Maybe thats the reason. Greets -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Wed, May 17, 2006 at 10:24:28PM +0200, Goswin von Brederlow wrote: What timeout? With feedback you would know exactly when it is done and wouldn't have to poll. Quoting Linus: : It really is very hard to accept the blocking behaviour. : : Some things can take a _loong_ time to be ready, including even : requiring user intervention. And even when scanning takes only a few : seconds, if there are multiple modules, you really want to scan things : in parallel. : : Not finding a disk is often a matter of timing out - not all buses : even have any real enumeration capability, and enumeration literally : ends up being try these addresses, and if nothing answers in 500 : msec, assume it's empty. : : Now, 500 msec may not sound very bad, but it all adds up. I get : unhappy if my bootup is a minute. I'd prefer booting up in a couple of : seconds. : : Also, how ready do you want things to be? Do you want to know the : device is there (disk at physical location X exists), or do you want : to have read the UUID off the disk and partitioned it? The latter is : what is needed for a mount, but it's often a _lot_ more expensive than : just knowing the disk is there, and it's not even necessarily needed : in many circumstances. So you can _not_ prevent polling. With some SCSI setups it literally takes minutes till all devices power up and answer to the inquiry command. If you happen to have several SCSI buses those latencies do add up pretty quickly. And not just SCSI: Linus: : For example, any USB host driver will always discover its devices : asynchronously, and has no dependency on CONFIG_HOTPLUG. It can take : several seconds for all the hubs to have powered up and discovered : what is behind them. Greg K-H: : And as others have pointed out, for a lot of busses, we just don't : know how long the device is going to take to be found. For one of my : boxes with a flaky USB controller, it takes _minutes_ for devices to : be detected (yes, it is broken hardware, but the rest of the system : works just fine while it is off and trying to be detected.) Scan the USB bus, find all devices on it, run all udev scripts for them, return. Greg K-H: : Another point is that some busses just don't know when all the devices : on it are found, as there is no such state. USB is one such bus, and : I imagine firewire is another one. So there is no real way for us to : delay at insmod time for all devices to be found. Just like the kernel always did prior to udev. You're missing a very important thing. This is _NOT_ a udev vs. pre-udev question. This is a new kernel hotplug behavior vs. old kernel hotplug behavior question. The fact is that the kernel behavior changed fundamentally in the early 2.6.x series compared to 2.4.x, and userspace did not follow that change ever since. udev simply exposes how the kernel really works. Not using udev solves NOTHING since it's not udev that is broken: it is the general userspace assumption about how the kernel works what is broken. I think it is pointless to bitch about udev without a clean proposal how should Debian handle hotplug events (including cases like when the device containing the root file system is literally plugged in _after_ the kernel/initrd has been loaded). That is a different matter. That never ever worked by itself and udev doesn't change anything there. Not different at _all_. As far as the kernel is concerned this is how it actually works right now. If you look at the initrd generators, they add delays and retries when mounting the root file system because the kernel does _not_ make any difference between delaying the discovery of an already-plugged-in device and physically plugging in the device later. In fact, if you have hotplug capable hardware, you could just connect the root device when you see the waiting for the root file system to appear (or similar) message and it would quite likely Just Work. But for already plugged in devices the device node was always ready when the insmod was done. And that has changed. And udev has _nothing_ to do with it apart from making that change more visible. Even with devfs. No. The confusion comes from the fact that when devfs was still maintained, the kernel still used the old device discovery scheme. If devfs had not been obsoloted it would have _exactly_ the same problems as udev because it used/uses the same internal in-kernel event notification mechanisms and data structures as udev. devfs may be somewhat faster than udev and therefore the time window between insmod returning and the device node appearing might be a bit smaller, but it would still be there and with a fast enough processor you'd hit it just as well you hit it with udev. For all cases where you load a module to activate a certain device (disk for initrd, mouse for X, ...) this is a serious step back. No, it's just the visible effect for the kernel slowly becoming fully hotplug-ready. It's just userspace that's
Re: debian and UDEV
On Thu, May 18, 2006 at 11:25:46AM +0200, Toni Mueller wrote: yes: Please say *why* newer 2.6.x kernels actually do depend on udev instead of hotplug. 1. They do not depend on it at all. 2.6.x kernels work just fine with a static /dev as long as you don't need dynamic device creation. There are no Depends: udev, nor even Recommends: udev or Suggests: udev in the kernel packages. 2. If you look at the upstream maintainers, you will find the same name in hotplug and udev. Greg K-H created udev to overcome the limitations of hotplug, and when it got ready, he abandoned hotplug. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Thu, May 18, 2006 at 04:16:33PM +0200, Gabor Gombas wrote: On Wed, May 17, 2006 at 10:24:28PM +0200, Goswin von Brederlow wrote: Just like the kernel always did prior to udev. You're missing a very important thing. This is _NOT_ a udev vs. pre-udev question. This is a new kernel hotplug behavior vs. old kernel hotplug behavior question. The fact is that the kernel behavior changed fundamentally in the early 2.6.x series compared to 2.4.x, and userspace did not follow that change ever since. udev simply exposes how the kernel really works. Not using udev solves NOTHING since it's not udev that is broken: it is the general userspace assumption about how the kernel works what is broken. This is pure crap. Install udev on my system - things break randomly and unreproducibly, due to race conditions all over the place. Install hotplug on my system, and make sure udev is not installed - things Just Work(TM). -- Fun will now commence -- Seven Of Nine, Ashes to Ashes, stardate 53679.4 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Gabor Gombas [EMAIL PROTECTED] writes: On Wed, May 17, 2006 at 10:24:28PM +0200, Goswin von Brederlow wrote: What timeout? With feedback you would know exactly when it is done and wouldn't have to poll. Quoting Linus: : It really is very hard to accept the blocking behaviour. : : Some things can take a _loong_ time to be ready, including even : requiring user intervention. And even when scanning takes only a few : seconds, if there are multiple modules, you really want to scan things : in parallel. : : Not finding a disk is often a matter of timing out - not all buses : even have any real enumeration capability, and enumeration literally : ends up being try these addresses, and if nothing answers in 500 : msec, assume it's empty. : : Now, 500 msec may not sound very bad, but it all adds up. I get : unhappy if my bootup is a minute. I'd prefer booting up in a couple of : seconds. : : Also, how ready do you want things to be? Do you want to know the : device is there (disk at physical location X exists), or do you want : to have read the UUID off the disk and partitioned it? The latter is : what is needed for a mount, but it's often a _lot_ more expensive than : just knowing the disk is there, and it's not even necessarily needed : in many circumstances. So you can _not_ prevent polling. With some SCSI setups it literally takes minutes till all devices power up and answer to the inquiry command. If you happen to have several SCSI buses those latencies do add up pretty quickly. Yes it does. But then it works all the time. Now the system randomly doesn't boot and instead of a 10 minute boot time you have a 3 hour drive to reset the system and analyse that is was just udev screwing you over. And not just SCSI: Linus: : For example, any USB host driver will always discover its devices : asynchronously, and has no dependency on CONFIG_HOTPLUG. It can take : several seconds for all the hubs to have powered up and discovered : what is behind them. Greg K-H: : And as others have pointed out, for a lot of busses, we just don't : know how long the device is going to take to be found. For one of my : boxes with a flaky USB controller, it takes _minutes_ for devices to : be detected (yes, it is broken hardware, but the rest of the system ^^ : works just fine while it is off and trying to be detected.) USB seems to be pretty broken by design. But most hardware is not. Even scsi, while taking some time to scan the bus, will finish in a reasonable time. Scan the USB bus, find all devices on it, run all udev scripts for them, return. Greg K-H: : Another point is that some busses just don't know when all the devices : on it are found, as there is no such state. USB is one such bus, and : I imagine firewire is another one. So there is no real way for us to : delay at insmod time for all devices to be found. Just like the kernel always did prior to udev. You're missing a very important thing. This is _NOT_ a udev vs. pre-udev question. This is a new kernel hotplug behavior vs. old kernel hotplug behavior question. The fact is that the kernel behavior changed fundamentally in the early 2.6.x series compared to 2.4.x, and userspace did not follow that change ever since. udev simply exposes how the kernel really works. Not using udev solves NOTHING since it's not udev that is broken: it is the general userspace assumption about how the kernel works what is broken. Yes, it is a problem with the kernels hotplug implementation. Hotplug and udev are just linked together. I'm not blaming the userspace part (udev) but the kernel part (hotplug). I think it is pointless to bitch about udev without a clean proposal how should Debian handle hotplug events (including cases like when the device containing the root file system is literally plugged in _after_ the kernel/initrd has been loaded). That is a different matter. That never ever worked by itself and udev doesn't change anything there. Not different at _all_. As far as the kernel is concerned this is how it actually works right now. If you look at the initrd generators, they add delays and retries when mounting the root file system because the kernel does _not_ make any difference between delaying the discovery of an already-plugged-in device and physically plugging in the device later. In fact, if you have hotplug capable hardware, you could just connect the root device when you see the waiting for the root file system to appear (or similar) message and it would quite likely Just Work. But for already plugged in devices the device node was always ready when the insmod was done. And that has changed. And udev has _nothing_ to do with it apart from making that change more visible. And that is what I consider broken. I know it is not going to change but I pain for all the users (and myself) that will (and already have been) get hit by
Re: debian and UDEV
Toni Mueller [EMAIL PROTECTED] writes: Hello, On Tue, 16.05.2006 at 04:05:41 +, Brian M. Carlson [EMAIL PROTECTED] wrote: [ udev being, umm, not very nice ] Need I say more? yes: Please say *why* newer 2.6.x kernels actually do depend on udev instead of hotplug. Thank you! Udev has just swallowed hotplug functionality making it obsolete. Nothing relay changed there. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Thu, May 18, 2006 at 04:50:44PM +0200, Wouter Verhelst wrote: Install udev on my system - things break randomly and unreproducibly, due to race conditions all over the place. This is quite in line with what I said about userspace not being ready for hotplug. The kernel can send the events in parallel (or nearly parallel on UP) and udev executes the events also in parallel. If the programs udev calls don't do the neccessary locking - fix them. Install hotplug on my system, and make sure udev is not installed - things Just Work(TM). That does not prove that the same race conditions do not exist. It only proves that the (quite noticable) overhead of forking a new shell for every event changes the timing so they are less likely to be hit, thus just papering over the bugs. Actually, if you look at the hotplug .agent scripts you may find artificial sleeps in some of them that's just seem to paper over the race conditions you're talking about... What happens if you add the same delays to the udev rules? Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences, Laboratory of Parallel and Distributed Systems Address : H-1132 Budapest Victor Hugo u. 18-22. Hungary Phone/Fax : +36 1 329-78-64 (secretary) W3: http://www.lpds.sztaki.hu - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On May 18, Gabor Gombas [EMAIL PROTECTED] wrote: Actually, if you look at the hotplug .agent scripts you may find artificial sleeps in some of them that's just seem to paper over the race conditions you're talking about... What happens if you add the same delays to the udev rules? RUN rules do not need delays, they are guaranteed to not be run until the device node is available. -- ciao, Marco signature.asc Description: Digital signature
Re: debian and UDEV
On Thu, May 18, 2006 at 05:01:53PM +0200, Goswin von Brederlow wrote: Now the system randomly doesn't boot and instead of a 10 minute boot time you have a 3 hour drive to reset the system and analyse that is was just udev screwing you over. Then introduce a timeout. Create /etc/default/wait_for_device_timeout and make it a policy that everyone must obey it. USB seems to be pretty broken by design. But most hardware is not. But normal users _do_ use broken hw, and they often has no choice. With the current kernel solution, broken hw presents a much smaller problem than it would with synchronous discovery. Even scsi, while taking some time to scan the bus, will finish in a reasonable time. But I don't want to wait for that. Users are unhappy because Linux already takes too long to boot. Yes, it is a problem with the kernels hotplug implementation. Hotplug and udev are just linked together. I'm not blaming the userspace part (udev) but the kernel part (hotplug). Why? Hotplug HW is now commodity, hw components appearing and disappearing any time (and in parallel) are a fact of life. Generalizing the concept so that _everything_ is treated as hot-plugged actually makes things easier since you no longer has to handle two separate cases. And that is what I consider broken. I know it is not going to change but I pain for all the users (and myself) that will (and already have been) get hit by problems caused by it. Then why not start working on a solution? There are several distros running udev, surely it would be possible to build a database of common problems and find solutions? Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Wouter Verhelst [EMAIL PROTECTED] writes: On Thu, May 18, 2006 at 04:16:33PM +0200, Gabor Gombas wrote: On Wed, May 17, 2006 at 10:24:28PM +0200, Goswin von Brederlow wrote: Just like the kernel always did prior to udev. You're missing a very important thing. This is _NOT_ a udev vs. pre-udev question. This is a new kernel hotplug behavior vs. old kernel hotplug behavior question. The fact is that the kernel behavior changed fundamentally in the early 2.6.x series compared to 2.4.x, and userspace did not follow that change ever since. udev simply exposes how the kernel really works. Not using udev solves NOTHING since it's not udev that is broken: it is the general userspace assumption about how the kernel works what is broken. This is pure crap. Install udev on my system - things break randomly and unreproducibly, due to race conditions all over the place. Install hotplug on my system, and make sure udev is not installed - things Just Work(TM). That is because udev is slower so the window of the race condition gets increased many many times. Without udev you don't have to wait for the mknod call to complete. And for most hardware that actualy makes the race condition disapear. Just broken stuff like usb that has asynchronous device detection it still breaks. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Gabor Gombas [EMAIL PROTECTED] writes: And that is what I consider broken. I know it is not going to change but I pain for all the users (and myself) that will (and already have been) get hit by problems caused by it. Then why not start working on a solution? There are several distros running udev, surely it would be possible to build a database of common problems and find solutions? Gabor Because the only _solution_ with current userspace is to kill the kernels hotplug design and go back to synchronous handling. And that is not going to happen. Anything else is just a work around that tries to hide the race conditions in current userspace environments. The only other solution that keeps the asynchronisity is what Joey suggested at the start: Instead of waiting/polling for udev events to finish move the code into udev rules that get called asynchronously when the devices appear. That means overhauling the complete boot concept of Debian. Not something I would do lightly. And not always easy. E.g. how do you convert startx into an udev rule so it can load the mouse modules savely? MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On May 18, Goswin von Brederlow [EMAIL PROTECTED] wrote: The only other solution that keeps the asynchronisity is what Joey suggested at the start: Instead of waiting/polling for udev events to finish move the code into udev rules that get called asynchronously when the devices appear. That means overhauling the complete boot This is what distributions like SuSE did, BTW. concept of Debian. Not something I would do lightly. And not always easy. E.g. how do you convert startx into an udev rule so it can load the mouse modules savely? You teach X about hotpluggable devices, which I understand is something that has been already planned. -- ciao, Marco signature.asc Description: Digital signature
Re: debian and UDEV
On Thu, May 18, 2006 at 07:16:00PM +0200, Goswin von Brederlow wrote: That is because udev is slower so the window of the race condition gets increased many many times. Without udev you don't have to wait for the mknod call to complete. I think you got the speed comparison wrong: udev does the mknod() from a C program running as a daemon, while hotplug means forking a new shell for every event. That should be several magnitudes slower. And for most hardware that actualy makes the race condition disapear. Just broken stuff like usb that has asynchronous device detection it still breaks. No, you're mixing things up. It seems that what you have a problem with is the dynamic /dev. Since hotplug does not provide a dynamic /dev, programs that assume /dev nodes are always there will of course work. Try adding UDEV_DISABLED=yes to /etc/udev.conf to disable the dynamic /dev (sorry, Marco :-) IMHO if you build a kernel with devfs or ndevfs enabled you'll have exactly the same races with hotplug than you have with udev. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences, Laboratory of Parallel and Distributed Systems Address : H-1132 Budapest Victor Hugo u. 18-22. Hungary Phone/Fax : +36 1 329-78-64 (secretary) W3: http://www.lpds.sztaki.hu - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Thu, May 18, 2006 at 07:22:47PM +0200, Goswin von Brederlow wrote:
The only other solution that keeps the asynchronisity is what Joey
suggested at the start: Instead of waiting/polling for udev events to
finish move the code into udev rules that get called asynchronously
when the devices appear.
Yep, that's the fix.
That means overhauling the complete boot concept of Debian.
Yes, it's just about time.
Not something I would do lightly. And not always easy.
Nobody said it will be. But somebody should start it.
E.g. how do you convert startx into an udev rule so it can load
the mouse modules savely?
By the time the user types his/her password and starts startx the mouse
will be surely detected (unless it is a broken USB device you said you
do not want to care about).
In case of {x|g|k}dm, by the time you sort out the boot process Xorg
will have input device hotplug so you can start it from the fbdev rule,
and the X server will start using the mouse automatically when it is
detected. In the mean time adding sleep 10 to /etc/init.d/{x|g|k}dm is
an acceptable workaround.
Gabor
--
-
MTA SZTAKI Computer and Automation Research Institute
Hungarian Academy of Sciences,
Laboratory of Parallel and Distributed Systems
Address : H-1132 Budapest Victor Hugo u. 18-22. Hungary
Phone/Fax : +36 1 329-78-64 (secretary)
W3: http://www.lpds.sztaki.hu
-
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Thu, May 18, 2006 at 07:45:01PM +0200, Marco d'Itri wrote: On May 18, Goswin von Brederlow [EMAIL PROTECTED] wrote: The only other solution that keeps the asynchronisity is what Joey suggested at the start: Instead of waiting/polling for udev events to finish move the code into udev rules that get called asynchronously when the devices appear. That means overhauling the complete boot This is what distributions like SuSE did, BTW. concept of Debian. Not something I would do lightly. And not always easy. E.g. how do you convert startx into an udev rule so it can load the mouse modules savely? You teach X about hotpluggable devices, which I understand is something that has been already planned. There's already a massively improved evdev driver written by our own Zephaniah E. Hull which allows this. There's been further work done by both Zephaniah and (also our own) Daniel Stone to implement this in the server. The new evdev driver is available and I'll be shipping it with 7.1 (possibly even before) while Daniel's patch is currently undergoing review and should be made available soon. Either way, the work is in progress and I'm personally really excited about it. - David Nusinow -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Gabor Gombas [EMAIL PROTECTED] writes: On Thu, May 18, 2006 at 07:16:00PM +0200, Goswin von Brederlow wrote: That is because udev is slower so the window of the race condition gets increased many many times. Without udev you don't have to wait for the mknod call to complete. I think you got the speed comparison wrong: udev does the mknod() from a C program running as a daemon, while hotplug means forking a new shell for every event. That should be several magnitudes slower. With a static dev there is no mknod syscall to wait for. That's what I was refering too. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Thu, May 18, 2006 at 07:16:00PM +0200, Goswin von Brederlow wrote: Wouter Verhelst [EMAIL PROTECTED] writes: On Thu, May 18, 2006 at 04:16:33PM +0200, Gabor Gombas wrote: On Wed, May 17, 2006 at 10:24:28PM +0200, Goswin von Brederlow wrote: Just like the kernel always did prior to udev. You're missing a very important thing. This is _NOT_ a udev vs. pre-udev question. This is a new kernel hotplug behavior vs. old kernel hotplug behavior question. The fact is that the kernel behavior changed fundamentally in the early 2.6.x series compared to 2.4.x, and userspace did not follow that change ever since. udev simply exposes how the kernel really works. Not using udev solves NOTHING since it's not udev that is broken: it is the general userspace assumption about how the kernel works what is broken. This is pure crap. Install udev on my system - things break randomly and unreproducibly, due to race conditions all over the place. Install hotplug on my system, and make sure udev is not installed - things Just Work(TM). That is because udev is slower so the window of the race condition gets increased many many times. Without udev you don't have to wait for the mknod call to complete. Which is a bad thing, why? And for most hardware that actualy makes the race condition disapear. Exactly. So there. -- Fun will now commence -- Seven Of Nine, Ashes to Ashes, stardate 53679.4 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Thu, May 18, 2006 at 08:05:35PM +0200, Gabor Gombas wrote: On Thu, May 18, 2006 at 07:16:00PM +0200, Goswin von Brederlow wrote: That is because udev is slower so the window of the race condition gets increased many many times. Without udev you don't have to wait for the mknod call to complete. I think you got the speed comparison wrong: udev does the mknod() from a C program running as a daemon, while hotplug means forking a new shell for every event. That should be several magnitudes slower. The difference being that hotplug doesn't even *attempt* to do an mknod() in its default configuration. Udev does do so, for everything. -- Fun will now commence -- Seven Of Nine, Ashes to Ashes, stardate 53679.4 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Gabor Gombas [EMAIL PROTECTED] writes:
On Thu, May 18, 2006 at 07:22:47PM +0200, Goswin von Brederlow wrote:
E.g. how do you convert startx into an udev rule so it can load
the mouse modules savely?
By the time the user types his/her password and starts startx the mouse
will be surely detected (unless it is a broken USB device you said you
do not want to care about).
Why should it be? Using the old style module loading on demand instead
of preloading everything with discover or friends there will be no
mouse device loaded. In that case X loads the mouse modules on start.
Before a sleep was added this always failed because of the race
condition between insmod and open of the device and udev creating the
device node.
In case of {x|g|k}dm, by the time you sort out the boot process Xorg
will have input device hotplug so you can start it from the fbdev rule,
and the X server will start using the mouse automatically when it is
detected. In the mean time adding sleep 10 to /etc/init.d/{x|g|k}dm is
an acceptable workaround.
It is a long way to go.
MfG
Goswin
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Am Donnerstag, 18. Mai 2006 19:22 schrieb Goswin von Brederlow: Because the only _solution_ with current userspace is to kill the kernels hotplug design and go back to synchronous handling. Another solution might be to dynamically attach to udev events. This is how in-kernel async device discovery is handled and correctly done, this is without race conditions: * register a function/program for new devices of your choice * load the proper module * your previously registers function will be run Don't know if this is already possible with udev alone. The current scheme to install files to run rules is too static than what many programs might need. The registration could be something like dbus or the like. The world is not synchronous and predictable. A fake synchronous thing in the kernel would only be a work-around, too. HS -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Hendrik Sattler [EMAIL PROTECTED] writes: Am Donnerstag, 18. Mai 2006 19:22 schrieb Goswin von Brederlow: Because the only _solution_ with current userspace is to kill the kernels hotplug design and go back to synchronous handling. Another solution might be to dynamically attach to udev events. This is how in-kernel async device discovery is handled and correctly done, this is without race conditions: * register a function/program for new devices of your choice * load the proper module * your previously registers function will be run But how do you detect when there is no device to be found to call the function? Say you have a scsi controler with no harddisks attached. You boot, you register a continue booting function for the scsi disks, load the scsi modules and then you hang. Bugger. So you have to add a timeout again which means a race condition or too long delay. You end up with polling and displaying waiting for / in a loop. People are saying this is a good thing because then you can hotplug your scsi disk or usb stick that has your / on it and it will continue booting. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Am Donnerstag, 18. Mai 2006 21:53 schrieb Goswin von Brederlow: Hendrik Sattler [EMAIL PROTECTED] writes: Am Donnerstag, 18. Mai 2006 19:22 schrieb Goswin von Brederlow: Because the only _solution_ with current userspace is to kill the kernels hotplug design and go back to synchronous handling. Another solution might be to dynamically attach to udev events. This is how in-kernel async device discovery is handled and correctly done, this is without race conditions: * register a function/program for new devices of your choice * load the proper module * your previously registers function will be run But how do you detect when there is no device to be found to call the function? That's of absolutely no interest because in this case, you cannot go on, having synchronous events or not. However, you might have alternatives for a root partition that are fulfilled with another function call. The point is that to keep the goal in mind: you want to achieve something e.g. by probing many different possibilities. Lets say you want to find a scanner: you probe SCSI busses, spawn a thread that test the parallel bus and another one for USB and whatever else. You only want to find one. You don't synchronize on any of those specific events but on the result: that a scanner is found. Sure, you need a timeout, but it is unrelated to a specific probing method Say you have a scsi controler with no harddisks attached. You boot, you register a continue booting function for the scsi disks, load the scsi modules and then you hang. Bugger. So you have to add a timeout again which means a race condition or too long delay. Just because you need synchronisation points does not mean that you have to do everything synchronously. The situation in userspace is completely the same as in kernel-space and also the solutions are the same. You end up with polling and displaying waiting for / in a loop. Yes. Why not. In fact, that's what your are interested in. You are NOT interested in finding all SCSI disks on the bus but only the device with the root partition. This is found at some time, but the bus scan might not be over, yet. And since you have your / now, you can go on without waiting for the bus scan to be finished. People are saying this is a good thing because then you can hotplug your scsi disk or usb stick that has your / on it and it will continue booting. But you only wait for the real goal. It is not a specific device file, that you are interested in but the root file system in a mounted state. This discussion is like explaining the principles of parallelism with threads and object oriented programming to a PASCAL programmer. HS pgpuIpw1J5YJQ.pgp Description: PGP signature
Re: debian and UDEV
Hendrik Sattler [EMAIL PROTECTED] writes: Am Donnerstag, 18. Mai 2006 21:53 schrieb Goswin von Brederlow: Hendrik Sattler [EMAIL PROTECTED] writes: Am Donnerstag, 18. Mai 2006 19:22 schrieb Goswin von Brederlow: Because the only _solution_ with current userspace is to kill the kernels hotplug design and go back to synchronous handling. Another solution might be to dynamically attach to udev events. This is how in-kernel async device discovery is handled and correctly done, this is without race conditions: * register a function/program for new devices of your choice * load the proper module * your previously registers function will be run But how do you detect when there is no device to be found to call the function? That's of absolutely no interest because in this case, you cannot go on, having synchronous events or not. However, you might have alternatives for a root partition that are fulfilled with another function call. The point is that to keep the goal in mind: you want to achieve something e.g. by probing many different possibilities. Lets say you want to find a scanner: you probe SCSI busses, spawn a thread that test the parallel bus and another one for USB and whatever else. You only want to find one. You don't synchronize on any of those specific events but on the result: that a scanner is found. Sure, you need a timeout, but it is unrelated to a specific probing method So you mean you register a function for block device found and then scan all the ide, sata, scsi, usb, ... drivers from the initrd and for every disk and partition the function gets called. When it gets called with the partition with the right signature (label or uuid for example) it mounts / and continues. Not a function for scsi disk ID 0 on scsi0 (3ware) found. I see what you mean. Keeping the function abstract enough allows removing a disk and plug it in somewhere else, e.g. change the scsi ID. Say you have a scsi controler with no harddisks attached. You boot, you register a continue booting function for the scsi disks, load the scsi modules and then you hang. Bugger. So you have to add a timeout again which means a race condition or too long delay. Just because you need synchronisation points does not mean that you have to do everything synchronously. The situation in userspace is completely the same as in kernel-space and also the solutions are the same. You end up with polling and displaying waiting for / in a loop. Yes. Why not. In fact, that's what your are interested in. You are NOT interested in finding all SCSI disks on the bus but only the device with the root partition. This is found at some time, but the bus scan might not be over, yet. And since you have your / now, you can go on without waiting for the bus scan to be finished. People are saying this is a good thing because then you can hotplug your scsi disk or usb stick that has your / on it and it will continue booting. But you only wait for the real goal. It is not a specific device file, that you are interested in but the root file system in a mounted state. This discussion is like explaining the principles of parallelism with threads and object oriented programming to a PASCAL programmer. HS Yeah, sorry. I wasn't thinking abstract enough. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Wed, May 17, 2006 at 12:19:35AM +0200, Goswin von Brederlow wrote: Matt Zimmerman [EMAIL PROTECTED] writes: You don't need to wait for a particular event to be finished processing; instead you should wait for the resource you actually need to become available, e.g. a device node. And how do you detect the resource not becoming available? With a timeout. Say I add a fsck script that runs before the final device node is created for any disk that gets added. That could take way longer than even the 3 minute timeout of the udevsettle. Why (and, for that matter, how) would you want to try to operate on the device before the device node is created? This seems rather contrived. It would be useful to add a simple utility for this to debianutils or similar so that it doesn't need to be reimplemented in so many places. I don't see any way to make this utility without having a race condition as it is. udev has no concept of this rule is finished afaik. It may be possible to check that all pending udev events have completed. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On May 17, Matt Zimmerman [EMAIL PROTECTED] wrote: It may be possible to check that all pending udev events have completed. It is possible to check the status of individual events by looking at the queue, but it's not obvious why people would want to do this instead of using RUN rules. -- ciao, Marco signature.asc Description: Digital signature
Re: debian and UDEV
Matt Zimmerman [EMAIL PROTECTED] writes: On Wed, May 17, 2006 at 12:19:35AM +0200, Goswin von Brederlow wrote: Matt Zimmerman [EMAIL PROTECTED] writes: You don't need to wait for a particular event to be finished processing; instead you should wait for the resource you actually need to become available, e.g. a device node. And how do you detect the resource not becoming available? With a timeout. == race condition. Say I add a fsck script that runs before the final device node is created for any disk that gets added. That could take way longer than even the 3 minute timeout of the udevsettle. Why (and, for that matter, how) would you want to try to operate on the device before the device node is created? This seems rather contrived. You create a temporary one in a private dir or one with private permissions. The later would probably screw your detection software totaly as the device appears but remains inaccessible till fsck is done. It would be useful to add a simple utility for this to debianutils or similar so that it doesn't need to be reimplemented in so many places. I don't see any way to make this utility without having a race condition as it is. udev has no concept of this rule is finished afaik. It may be possible to check that all pending udev events have completed. Which is still stupid not to have in the kernel API as feedback from the event manager and have insmod optionaly block. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
The rest of the Linux distribution world is using udev with the current semantics and has not crumbled. If you don't like the current semantics, I understand, but that shouldn't stand in the way of its adoption. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Gabor Gombas [EMAIL PROTECTED] writes: On Wed, May 17, 2006 at 10:44:21AM +0200, Goswin von Brederlow wrote: Which is still stupid not to have in the kernel API as feedback from the event manager and have insmod optionaly block. For that to work you should make device discovery synchronous everywhere in the kernel. That would be a big change, and then people would start complaining that the boot process suddenly takes 10-20 minutes instead of 10-20 seconds (because the kernel now has to wait for time-out for non-existing devices which you never noticed before thanks to the asynchronous discovery). What timeout? With feedback you would know exactly when it is done and wouldn't have to poll. If you mean the timeout of the actual device drivers themself then that is what pre udev kernels have always done and I think kernels still do anway. If you load a scsi driver module it will block to scan the scsi bus, generate the hotplug events and only then go asynchronous. If you mean running the udev scripts then yes, that would get linearised instead of running in parallel to other module loads. But then again mknod should not take that long. And what do you want to do for hot-pluggable devices like USB or SATA? Should insmod block forever if there is a free SATA port just in case the sysadmin wants to plug in a disk later? And if not, why is this case different from insmod returning immediately and reporting all plugged-in disks later? You _must_ be able to handle the disk-plugged-in-later case and that should just work for the already-plugged-in disks as well... Scan the USB bus, find all devices on it, run all udev scripts for them, return. Just like the kernel always did prior to udev. I think it is pointless to bitch about udev without a clean proposal how should Debian handle hotplug events (including cases like when the device containing the root file system is literally plugged in _after_ the kernel/initrd has been loaded). That is a different matter. That never ever worked by itself and udev doesn't change anything there. But for already plugged in devices the device node was always ready when the insmod was done. Even with devfs. With udev the device node appears at some random time after insmod has returned. For all cases where you load a module to activate a certain device (disk for initrd, mouse for X, ...) this is a serious step back. Gabor MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Wed, May 17, 2006 at 10:44:21AM +0200, Goswin von Brederlow wrote: Which is still stupid not to have in the kernel API as feedback from the event manager and have insmod optionaly block. For that to work you should make device discovery synchronous everywhere in the kernel. That would be a big change, and then people would start complaining that the boot process suddenly takes 10-20 minutes instead of 10-20 seconds (because the kernel now has to wait for time-out for non-existing devices which you never noticed before thanks to the asynchronous discovery). And what do you want to do for hot-pluggable devices like USB or SATA? Should insmod block forever if there is a free SATA port just in case the sysadmin wants to plug in a disk later? And if not, why is this case different from insmod returning immediately and reporting all plugged-in disks later? You _must_ be able to handle the disk-plugged-in-later case and that should just work for the already-plugged-in disks as well... I think it is pointless to bitch about udev without a clean proposal how should Debian handle hotplug events (including cases like when the device containing the root file system is literally plugged in _after_ the kernel/initrd has been loaded). Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Matt Zimmerman [EMAIL PROTECTED] writes: On Mon, May 15, 2006 at 12:14:48PM +0200, Goswin von Brederlow wrote: Matt Zimmerman [EMAIL PROTECTED] writes: I don't see it as a general issue either; if you have problems of this type, you should report them to the bug tracking system so that they can be fixed. Debian as an organization can't hope to test all of the hardware available in the market, so we rely on feedback from folks such as yourself to let us know when there is a problem. The most hideous problem of hotplug/udev is the following: You can't wait for an hotplug/udev event to be done processing. That is always done asynchron without any feedback of completion. You don't need to wait for a particular event to be finished processing; instead you should wait for the resource you actually need to become available, e.g. a device node. And how do you detect the resource not becoming available? Say I add a fsck script that runs before the final device node is created for any disk that gets added. That could take way longer than even the 3 minute timeout of the udevsettle. It would be useful to add a simple utility for this to debianutils or similar so that it doesn't need to be reimplemented in so many places. I don't see any way to make this utility without having a race condition as it is. udev has no concept of this rule is finished afaik. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Mon, May 15, 2006 at 12:14:48PM +0200, Goswin von Brederlow wrote: Matt Zimmerman [EMAIL PROTECTED] writes: I don't see it as a general issue either; if you have problems of this type, you should report them to the bug tracking system so that they can be fixed. Debian as an organization can't hope to test all of the hardware available in the market, so we rely on feedback from folks such as yourself to let us know when there is a problem. The most hideous problem of hotplug/udev is the following: You can't wait for an hotplug/udev event to be done processing. That is always done asynchron without any feedback of completion. You don't need to wait for a particular event to be finished processing; instead you should wait for the resource you actually need to become available, e.g. a device node. It would be useful to add a simple utility for this to debianutils or similar so that it doesn't need to be reimplemented in so many places. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
Matt Zimmerman wrote: On Mon, May 15, 2006 at 12:14:48PM +0200, Goswin von Brederlow wrote: Matt Zimmerman [EMAIL PROTECTED] writes: I don't see it as a general issue either; if you have problems of this type, you should report them to the bug tracking system so that they can be fixed. Debian as an organization can't hope to test all of the hardware available in the market, so we rely on feedback from folks such as yourself to let us know when there is a problem. The most hideous problem of hotplug/udev is the following: You can't wait for an hotplug/udev event to be done processing. That is always done asynchron without any feedback of completion. You don't need to wait for a particular event to be finished processing; instead you should wait for the resource you actually need to become available, e.g. a device node. It would be useful to add a simple utility for this to debianutils or similar so that it doesn't need to be reimplemented in so many places. Why not use udevsettle that comes with new udev just for this purpose? -- Alexander E. Patrakov -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debian and UDEV
On Tue, 2006-05-16 at 08:52 +0600, Alexander E. Patrakov wrote: Matt Zimmerman wrote: You don't need to wait for a particular event to be finished processing; instead you should wait for the resource you actually need to become available, e.g. a device node. It would be useful to add a simple utility for this to debianutils or similar so that it doesn't need to be reimplemented in so many places. Why not use udevsettle that comes with new udev just for this purpose? Probably because: * you'd have to depend on udev (= 0.090-1), which: - is not Essential: yes, unlike debianutils - means your code will no longer work on 2.4 kernels, or non-Linux ports - means that if udev ever goes away (we hope) your package will FTBFS - means your code will no longer be backportable - means that your package will have a dependency on a package that breaks frequently[0] - means that if any version of udev stops working with your package, you will have to conflict with it * man 8 udevsettle doesn't tell you: - that the default timeout is *three minutes*, which is a totally unreasonable default for obvious reasons - what the return codes are in case of success, failure, or timeout * whether the event queue has settled tells you nothing about whether a resource is available Need I say more? [0] No other package, to my knowledge, has the same combination of being essential for booting the system, huge numbers of high severity bugs, gratuitous incompatibility with older kernels, and frequent bugs on non-i386 architectures. signature.asc Description: This is a digitally signed message part
