Re: PolicyKit confusion
On Sat, Dec 24, 2011 at 10:44:53PM +1000, Da Rock wrote: > > And yes, that will block write calls until they're truely done. > You said that was slower too. A bit. But I think it is a good trade-off for filesystems on USB disks. It certainly circumvents a whole lot of problems with filesystem consistency and stuff. > All valid points. What it boils down to is if you yank the power or the > disk during write you're screwed no matter what fancy tricks you play- > include hal/dbus/blah! bull crap. Yep. :-) > You have tickled my imaginationings again, and I have an idea to throw > at the gurus on the list (including you Roland): Assuming null/void > computer literacy, and the use of a DE; what if one had an (can't be a > script can it - pointless. Thinking on the fly here...) applet? dock > notifier? that indicated the disk was in use and not to remove - > primarily for a "blinkered" user that is missing the flashing led on the > drive (or one missing an led)? Stays on top, flashing on the screen. Use > mount -d to test disk status, then lets you know that it can be removed. > I still have to test this, but I thought I'd throw it out there. Just add a button to that applet that is called "eject" or "remove" (or something like that) and have that unmount the fs in question. Then put up a dialog or notification saying "The USB device can be removed" or something of that nature. That's basically what windows does, and it works well enough. > Basically, the situation (that I have found myself in numerous times) is > the write stops to the disk because the buffers need to reach a certain > level before write occurs, or umount. So one can mistakenly assume that > all is kocher and find no file when they reinsert. Files are lost that > way, and at the least its annoying. That can still even occur with hal > and blah blah blah. I'd still advocate mounting these discs with '-o sync'. Yes, it is a bit slower but it prevents a lot of trouble and is a lot easier. > Is this feasible though? If the user still can't pay attention to _all_ > of that then they're not worth worrying about- can't be holding their > hand the whole time :) Then, using devd, one can implement the > mount/umount scripts with a quick fsck on insertion to be sure ICIE (In > Case of Idiot Event). Unmounting with devd is just triage. Umount cannot do some useful things (like flushing buffers) if the disk has already been removed! It can only remove the mount so you won't get trouble when the user sticks the device back in. And on a 4 GiB thumbdrive, I doubt an fsck is going to be really quick! Remember that the drive will be basically unusable when the check is in progress. I could imagine getting all kinds of complaints from users in the shape of "I plugged it in and it doesn't work". It should not be too difficult to write a script that is called from devd when a new USB device is detected (I think you should watch for CREATE events from the CDEV subsystem of the DEVFS system). I think it should check /dev first for new disk labels (/dev/msdosfs or /dev/ufs) and second for for disk devices, and try and mount them using appropriate filesystems under /mnt/$USER/$NUM (the latter because there might be more than one USB device in use). Optionally it could also launch a new instance or new window of the user's file manager, depending on how much hand-holding and customization you want to do. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpYMMD1iv2hW.pgp Description: PGP signature
Re: PolicyKit confusion
On 12/24/11 22:21, Roland Smith wrote: On Sat, Dec 24, 2011 at 12:03:30PM +1000, Da Rock wrote: might render the application performing it unresponsive during that time. Bingo! Thats exactly the issue. Taking a look at FUSE I can see there are calls that FUSE_FSYNC which I would say is the closest to what is needed. The question is are they implemented in say fuse_msdosfs (I may have just barfed that, but you get the idea) and maybe some others? Use the source, Luke! ;-) Believe me, I'm checking now. A lot of code though, if someone already knows the answer :) That way the filesystem is treated as normal, the app writes and can return happily, and FUSE takes the responsibility to ensure the data continues to write until finished instead of holding it in buffer for an indeterminate period leaving the user thinking alls well (ergo, the light continues flashing). I'm not sure how long the defaults waits are. Normally the systems tries to cluster writes sequenstially for improved throughput. But I'm guessing we're talking about fractions of seconds at most. Again, browsing the source might be useful. Also chapter 6 of "The Design and Implementation of the FreeBSD operating system" might provide insight. I'm basing my logic on experience and not theory at this point, although I have read through most of the developer docs and studied OS design. I think it waits until it reckons it has enough to fill a sector or something, buffer full; at any rate it will halt and not do a damned thing until you tell it umount and then it goes crazy- up to a minute sometimes. This is my point about the led: a user could easily mistake this for the write to be finished and yank it when there is no file there (the one they want anyway)- then end up with a "dirty" filesystem. Its very misleading. Failing that, how does one ensure data is synced manually? Is there a call that could be run periodically at a shorter, more regular intervals? The sync(1) program flushes the system's write cache. Yahh... that is supposed to be before the system halts- right? So how would it be used here? Nothing in the man says how it is used, and if it is used its system wide and not focused to a particular filesystem, whatever. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: PolicyKit confusion
On 12/24/11 22:08, Roland Smith wrote: On Sat, Dec 24, 2011 at 12:23:17PM +1000, Da Rock wrote: On 12/24/11 11:34, Roland Smith wrote: On Sat, Dec 24, 2011 at 09:57:38AM +1000, Da Rock wrote: FreeBSD be default already does buffering in the VFS layer (unless you turn that off). I don't think that adding more buffering would help. It might even make matters worse. If data is buffered and not immediately written to the USB stick, it will show no activity. This might even give the user a false impression it is finished... That there is exactly the problem. Any way to prevent that though? Yes. Using the '-o sync' option with mount. To the best of my understanding that means that a write action will be executed immediately and that write(2) will not return until it is finished. Just discovered something: what about async as an option? The major problem with async is on UFS+SU - the SU's get in the road and can I've had problems with filesystems becoming inconsistent with softupdates. I've disabled them on most filesystems. result in inconsistencies. But vfat is another kettle of fish altogether. The mount(8) manual warns that async is dangerous because it doesn't guarantee that the fs structure on disk stays consistent. The other side of the coin is (as you say) that vfat doesn't have much of a structure. :-) The long and the short of it is that async is bad for SU filesystems- others *should* be reasonably ok. I'd have to check on NTFS. I just had a brainwave and looked it up, after a google or two and reading the mount_msdosfs man page it is possible; but is it a solution? The writes are done sequentially (I think), and the app can move on while the system writes the disk. Unless I'm missing something here... In my script to mount USB drives I use the following options for mount_msdosfs: "-o noatime -o sync -o noexec -o nosuid" And yes, that will block write calls until they're truely done. You said that was slower too. But OTOH, if you use async, an umount will block until all data is written. So it is a question of waiting now or waiting later. ;-) Personally I like the security and consistency that -o sync brings. Since I mostly use cp from an xterm to copy things to/from USB disks, it doesn't bother me when is stays busy a while longer. All valid points. What it boils down to is if you yank the power or the disk during write you're screwed no matter what fancy tricks you play- include hal/dbus/blah! bull crap. The main difference is the use of async or sync: you either screw up with one file or a few. You have tickled my imaginationings again, and I have an idea to throw at the gurus on the list (including you Roland): Assuming null/void computer literacy, and the use of a DE; what if one had an (can't be a script can it - pointless. Thinking on the fly here...) applet? dock notifier? that indicated the disk was in use and not to remove - primarily for a "blinkered" user that is missing the flashing led on the drive (or one missing an led)? Stays on top, flashing on the screen. Use mount -d to test disk status, then lets you know that it can be removed. I still have to test this, but I thought I'd throw it out there. Basically, the situation (that I have found myself in numerous times) is the write stops to the disk because the buffers need to reach a certain level before write occurs, or umount. So one can mistakenly assume that all is kocher and find no file when they reinsert. Files are lost that way, and at the least its annoying. That can still even occur with hal and blah blah blah. Is this feasible though? If the user still can't pay attention to _all_ of that then they're not worth worrying about- can't be holding their hand the whole time :) Then, using devd, one can implement the mount/umount scripts with a quick fsck on insertion to be sure ICIE (In Case of Idiot Event). ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: PolicyKit confusion
On Sat, Dec 24, 2011 at 12:03:30PM +1000, Da Rock wrote: > > might render the application performing it unresponsive during that time. > Bingo! Thats exactly the issue. Taking a look at FUSE I can see there > are calls that FUSE_FSYNC which I would say is the closest to what is > needed. The question is are they implemented in say fuse_msdosfs (I may > have just barfed that, but you get the idea) and maybe some others? Use the source, Luke! ;-) > That > way the filesystem is treated as normal, the app writes and can return > happily, and FUSE takes the responsibility to ensure the data continues > to write until finished instead of holding it in buffer for an > indeterminate period leaving the user thinking alls well (ergo, the > light continues flashing). I'm not sure how long the defaults waits are. Normally the systems tries to cluster writes sequenstially for improved throughput. But I'm guessing we're talking about fractions of seconds at most. Again, browsing the source might be useful. Also chapter 6 of "The Design and Implementation of the FreeBSD operating system" might provide insight. > Failing that, how does one ensure data is synced manually? Is there a > call that could be run periodically at a shorter, more regular intervals? The sync(1) program flushes the system's write cache. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpG1NvZ0ce9Y.pgp Description: PGP signature
Re: PolicyKit confusion
On Sat, Dec 24, 2011 at 12:23:17PM +1000, Da Rock wrote: > On 12/24/11 11:34, Roland Smith wrote: > > On Sat, Dec 24, 2011 at 09:57:38AM +1000, Da Rock wrote: > >>> FreeBSD be default already does buffering in the VFS layer (unless you > >>> turn > >>> that off). I don't think that adding more buffering would help. It might > >>> even > >>> make matters worse. If data is buffered and not immediately written to > >>> the USB > >>> stick, it will show no activity. This might even give the user a false > >>> impression it is finished... > >> That there is exactly the problem. Any way to prevent that though? > > Yes. Using the '-o sync' option with mount. To the best of my understanding > > that means that a write action will be executed immediately and that > > write(2) > > will not return until it is finished. > Just discovered something: what about async as an option? The major > problem with async is on UFS+SU - the SU's get in the road and can I've had problems with filesystems becoming inconsistent with softupdates. I've disabled them on most filesystems. > result in inconsistencies. But vfat is another kettle of fish altogether. The mount(8) manual warns that async is dangerous because it doesn't guarantee that the fs structure on disk stays consistent. The other side of the coin is (as you say) that vfat doesn't have much of a structure. :-) > I just had a brainwave and looked it up, after a google or two and > reading the mount_msdosfs man page it is possible; but is it a solution? > The writes are done sequentially (I think), and the app can move on > while the system writes the disk. Unless I'm missing something here... In my script to mount USB drives I use the following options for mount_msdosfs: "-o noatime -o sync -o noexec -o nosuid" And yes, that will block write calls until they're truely done. But OTOH, if you use async, an umount will block until all data is written. So it is a question of waiting now or waiting later. ;-) Personally I like the security and consistency that -o sync brings. Since I mostly use cp from an xterm to copy things to/from USB disks, it doesn't bother me when is stays busy a while longer. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpT8Kb65fhZt.pgp Description: PGP signature
Re: PolicyKit confusion
On 12/24/11 11:34, Roland Smith wrote: On Sat, Dec 24, 2011 at 09:57:38AM +1000, Da Rock wrote: FreeBSD be default already does buffering in the VFS layer (unless you turn that off). I don't think that adding more buffering would help. It might even make matters worse. If data is buffered and not immediately written to the USB stick, it will show no activity. This might even give the user a false impression it is finished... That there is exactly the problem. Any way to prevent that though? Yes. Using the '-o sync' option with mount. To the best of my understanding that means that a write action will be executed immediately and that write(2) will not return until it is finished. Just discovered something: what about async as an option? The major problem with async is on UFS+SU - the SU's get in the road and can result in inconsistencies. But vfat is another kettle of fish altogether. I just had a brainwave and looked it up, after a google or two and reading the mount_msdosfs man page it is possible; but is it a solution? The writes are done sequentially (I think), and the app can move on while the system writes the disk. Unless I'm missing something here... I've had some experience with async and gjournal, so it should work as I'm dreaming it :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: PolicyKit confusion
On 12/24/11 11:34, Roland Smith wrote: On Sat, Dec 24, 2011 at 09:57:38AM +1000, Da Rock wrote: FreeBSD be default already does buffering in the VFS layer (unless you turn that off). I don't think that adding more buffering would help. It might even make matters worse. If data is buffered and not immediately written to the USB stick, it will show no activity. This might even give the user a false impression it is finished... That there is exactly the problem. Any way to prevent that though? Yes. Using the '-o sync' option with mount. To the best of my understanding that means that a write action will be executed immediately and that write(2) will not return until it is finished. It is a heck of a problem. Distributed filesystems like Coda [http://en.wikipedia.org/wiki/Coda_%28file_system%29] might offer a partial solution. But it doesn't seem like they're being widely used. The only sane way to handle this is for the application to get an error from the next write reporting that the filesystem has disappeared. Which it should then report to the user because that's the person that pulled the plug, so to speak. Man, What a mess! The real solution is to keep the light flashing until all the data written to disk. The abovementioned -o sync option will do that for you. The price is that the write syscall will block until it is finished. This might render the application performing it unresponsive during that time. Bingo! Thats exactly the issue. Taking a look at FUSE I can see there are calls that FUSE_FSYNC which I would say is the closest to what is needed. The question is are they implemented in say fuse_msdosfs (I may have just barfed that, but you get the idea) and maybe some others? That way the filesystem is treated as normal, the app writes and can return happily, and FUSE takes the responsibility to ensure the data continues to write until finished instead of holding it in buffer for an indeterminate period leaving the user thinking alls well (ergo, the light continues flashing). Failing that, how does one ensure data is synced manually? Is there a call that could be run periodically at a shorter, more regular intervals? Either that or something that just tells the disk to keep flashing its little led while the buffer is not empty :) All the mayor players in this drama, hal, policykit and dbus are maintained by gnome@. In practice that _might_ mean that no single person cares enough to care and feed them. Ahh. Now that may explain some things. But by your meaning are you talking about the software development itself or the developers? LOL What I mean here is that there are no dedicated maintainers for the FreeBSD ports. When that is the case, that maintainer address of a port is usually that of a relevant mailing list. That way problems with such a port at least get the attention of people with relevant interests. As for the original developers, who knows? IMNSHO their solutions look overly complex, Indeed. At least from a BSD perspective, anyway. Linux is whole other animal... family, xenotype even :) Too many chiefs and not enough indians over there I reckon. In my mind reading your comments I was imagining some starving, malnourished developers trying to build these complex software designs surrounded by some Doc Brown-like inventions in the dungeons of Novell or something... :) Hell, it could even be the dungeons of Microsoft, with Gates uttering "Now I'll get you all, and your little dog too.." :D And I'm kind of allergic to desktop environments. Over the years I've tried several window managers, but I keep coming back to good old FVWM. :-) Me too- can't get away from icewm. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: PolicyKit confusion
On Sat, Dec 24, 2011 at 09:57:38AM +1000, Da Rock wrote: > > FreeBSD be default already does buffering in the VFS layer (unless you turn > > that off). I don't think that adding more buffering would help. It might > > even > > make matters worse. If data is buffered and not immediately written to the > > USB > > stick, it will show no activity. This might even give the user a false > > impression it is finished... > That there is exactly the problem. Any way to prevent that though? Yes. Using the '-o sync' option with mount. To the best of my understanding that means that a write action will be executed immediately and that write(2) will not return until it is finished. > No, I hadn't considered that scenario. Thats why these lists provide a > great sounding board :) It is a heck of a problem. Distributed filesystems like Coda [http://en.wikipedia.org/wiki/Coda_%28file_system%29] might offer a partial solution. But it doesn't seem like they're being widely used. > > The only sane way to handle this is for the application to get an error from > > the next write reporting that the filesystem has disappeared. Which it > > should > > then report to the user because that's the person that pulled the plug, so > > to > > speak. > Man, What a mess! The real solution is to keep the light flashing until > all the data written to disk. The abovementioned -o sync option will do that for you. The price is that the write syscall will block until it is finished. This might render the application performing it unresponsive during that time. > > All the mayor players in this drama, hal, policykit and dbus are maintained > > by > > gnome@. In practice that _might_ mean that no single person cares enough to > > care and feed them. > > > Ahh. Now that may explain some things. But by your meaning are you > talking about the software development itself or the developers? LOL What I mean here is that there are no dedicated maintainers for the FreeBSD ports. When that is the case, that maintainer address of a port is usually that of a relevant mailing list. That way problems with such a port at least get the attention of people with relevant interests. As for the original developers, who knows? IMNSHO their solutions look overly complex, but I haven't looked at the problem they try to solve because it's not a problem that bothers me. And I'm kind of allergic to desktop environments. Over the years I've tried several window managers, but I keep coming back to good old FVWM. :-) Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgppjv9MeNctZ.pgp Description: PGP signature
Re: PolicyKit confusion
On 12/24/11 09:21, Roland Smith wrote: On Sat, Dec 24, 2011 at 08:03:22AM +1000, Da Rock wrote: If the network goes down, network drives won't work. Your users will be sad/scared/frustrated with or without error messages, I'm guessing. Nah. They'd flip out a whole lot more when the screen literally fills with error messages and keeps going. Frustrated they can handle and maybe complain, but that would make them run away... :) Wouldn't you get some error message or other if a network drive becomes unreachable no matter what? True, but its not so bad as a screenful- and I mean a screenful! Another way to go about it is to install e.g. ubuntu on a virtual machine and peek under the hood how it works there. But as you say it's probably tied into udev pretty tightly. Tried that too, but each distro has there own "hack" to make it work for them. Crazy huh? I tried ubuntu once in a VM (I was a slackware user before moving to FreeBSD). Had a quick look with ps and was rather appalled at all the stuff that was running with no obvious way to turn it off. Devd just gets some notifications and acts on them. There is a problem with mounted usb devices, but that is one of architecture, I guess. Devd only gets notified _after_ a device has been pulled. There is no way you can prevent data loss in all cases like that. On windows you're supposed to "prepare to eject" a USB device before pulling it out as well. The only "cure" is to mount a device syncronously, and disable _all_ write caching for those devices. If you try that you'll find that doing so has significant performance impact and not in a good way (disks are slw). Almost need a "journaling" system for them. Any thoughts? What about setting up a temp folder (non-volatile buffer?) and a sync? Track devices using the uuid label? With proper mount settings and synchronous writes you might be able to prevent most damage, but it'll be slow. The default for mount is to write metadata syncronously, while data I/O is done async, see mount(8). No matter what you do, if a user pulls a USB stick during a write, the filesystem on it will be left in an inconsistent state. Nothing you can do about that. True. I'm only thinking of when the little light stops flashing. FreeBSD be default already does buffering in the VFS layer (unless you turn that off). I don't think that adding more buffering would help. It might even make matters worse. If data is buffered and not immediately written to the USB stick, it will show no activity. This might even give the user a false impression it is finished... That there is exactly the problem. Any way to prevent that though? Basically if a USB drive is plugged back in again, you have to accept the state that it is in at that time. You cannot assume that it's state is still the same or even related as the last time it was plugged in. But suppose for the sake of the argument that you have a complete and correct copy of the USB stick's filesystem at the time it was pulled buffered. Now assume the same device is plugged in again. You read the complete state of the filesystem and compare it to your buffer. Suppose there is a difference between the two. What are you going to do? Without further information there is no way of knowing which of the changes are OK because they were done e.g. on another computer. No, I hadn't considered that scenario. Thats why these lists provide a great sounding board :) And there is the application that is writing that is to be considered. With buffering enabled, write system calls return as soon as the metadata is written and the data is queued, IIRC. So as fas as the app is concerned, it is done. Of course the next system call to write to the same fs after it is pulled will get an error. But that still leaves the application's image of the file's state different from reality. The only sane way to handle this is for the application to get an error from the next write reporting that the filesystem has disappeared. Which it should then report to the user because that's the person that pulled the plug, so to speak. Man, What a mess! The real solution is to keep the light flashing until all the data written to disk. As a user, and watching other less literate users, thats what they're all watching for. It seems pretty simple doesn't it? Does fuse help this at all? Most are vfat or ntfs after all... If they're not, then they'd be a bit more literate anyway, surely. And if one was really anal a fuse-ufs :) submit it to the freebsd-doc mailing list for inclusion in the official docs? I may yet do that, but in the interim I'm going to get around to writing up my findings on a lot of different aspects of the systems in a wiki (I'll put up my findings on those as well...). Maybe my pain can help someone else :) Please but up a link to that wiki. :-) Absolutely. I'm hoping to find answers to the more obscure settings. I'm writing how to's or tutorials- just the variables and conf settings and t
Re: PolicyKit confusion
On Sat, Dec 24, 2011 at 08:03:22AM +1000, Da Rock wrote: > > If the network goes down, network drives won't work. Your users will be > > sad/scared/frustrated with or without error messages, I'm guessing. > Nah. They'd flip out a whole lot more when the screen literally fills > with error messages and keeps going. Frustrated they can handle and > maybe complain, but that would make them run away... :) Wouldn't you get some error message or other if a network drive becomes unreachable no matter what? > > Another way to go about it is to install e.g. ubuntu on a virtual machine > > and > > peek under the hood how it works there. But as you say it's probably tied > > into > > udev pretty tightly. > Tried that too, but each distro has there own "hack" to make it work for > them. Crazy huh? I tried ubuntu once in a VM (I was a slackware user before moving to FreeBSD). Had a quick look with ps and was rather appalled at all the stuff that was running with no obvious way to turn it off. > > Devd just gets some notifications and acts on them. There is a problem with > > mounted usb devices, but that is one of architecture, I guess. Devd only > > gets > > notified _after_ a device has been pulled. There is no way you can prevent > > data loss in all cases like that. On windows you're supposed to "prepare to > > eject" a USB device before pulling it out as well. The only "cure" is to > > mount > > a device syncronously, and disable _all_ write caching for those devices. If > > you try that you'll find that doing so has significant performance impact > > and > > not in a good way (disks are slw). > Almost need a "journaling" system for them. Any thoughts? What about > setting up a temp folder (non-volatile buffer?) and a sync? Track > devices using the uuid label? With proper mount settings and synchronous writes you might be able to prevent most damage, but it'll be slow. The default for mount is to write metadata syncronously, while data I/O is done async, see mount(8). No matter what you do, if a user pulls a USB stick during a write, the filesystem on it will be left in an inconsistent state. Nothing you can do about that. FreeBSD be default already does buffering in the VFS layer (unless you turn that off). I don't think that adding more buffering would help. It might even make matters worse. If data is buffered and not immediately written to the USB stick, it will show no activity. This might even give the user a false impression it is finished... Basically if a USB drive is plugged back in again, you have to accept the state that it is in at that time. You cannot assume that it's state is still the same or even related as the last time it was plugged in. But suppose for the sake of the argument that you have a complete and correct copy of the USB stick's filesystem at the time it was pulled buffered. Now assume the same device is plugged in again. You read the complete state of the filesystem and compare it to your buffer. Suppose there is a difference between the two. What are you going to do? Without further information there is no way of knowing which of the changes are OK because they were done e.g. on another computer. And there is the application that is writing that is to be considered. With buffering enabled, write system calls return as soon as the metadata is written and the data is queued, IIRC. So as fas as the app is concerned, it is done. Of course the next system call to write to the same fs after it is pulled will get an error. But that still leaves the application's image of the file's state different from reality. The only sane way to handle this is for the application to get an error from the next write reporting that the filesystem has disappeared. Which it should then report to the user because that's the person that pulled the plug, so to speak. > > submit it to the freebsd-doc mailing list for inclusion in the official > > docs? > I may yet do that, but in the interim I'm going to get around to writing > up my findings on a lot of different aspects of the systems in a wiki > (I'll put up my findings on those as well...). Maybe my pain can help > someone else :) Please but up a link to that wiki. :-) > For reference _all_ my systems are FreeBSD: from laptops/desktops, > HTPC's and servers. I'd like to be able to show a system better and more > robust than the alternatives out there as well as easy on the users, and > thats what I'm always working towards. Nice! I wish I was able to use FreeBSD at work (other than on my own laptop). > > And talking about mailing lists, maybe you should try your luck on the > > freebsd-gnome list? > > [http://lists.freebsd.org/mailman/listinfo/freebsd-gnome] > I would but I'm not subscribed to that one (must be about the only one > I'm not on :) ), and it hadn't come to mind as I wasn't using gnome! I'm > using nautilus for testing as it has more features, but I'm intending on > using pcmanfm or similar- lightweight, b
Re: PolicyKit confusion
On 12/24/11 03:31, Roland Smith wrote: On Sat, Dec 24, 2011 at 01:27:23AM +1000, Da Rock wrote: couldn't give a shit about network drives. One scenario is network goes down and I get a screenful of error messages- it annoys the shit out of me, let alone scaring illiterate users. If the network goes down, network drives won't work. Your users will be sad/scared/frustrated with or without error messages, I'm guessing. Nah. They'd flip out a whole lot more when the screen literally fills with error messages and keeps going. Frustrated they can handle and maybe complain, but that would make them run away... :) I don't think enough people care to make it really work under FreeBSD. I've certainly never missed it. They don't care too much under linux either So many years, yet not a single user doc anywhere on any of the f***ing monsters! Goddamn linux devs- ever heard of KISS?! How about a user doc for something so bloody confusing it takes a brain surgeon to figure it out? That's exactly why I've avoided using this stuff. :-) Me too. I just made do with what was there. Ahh, one more thing of note here: polkit-gnome-authorization would not work under any circumstances (run as root or otherwise) to change policies! WTF! You might take a look at devd(8) as a FreeBSD alternative, but I'm not sure if it notices new da devices popping up. Oh, believe me I'd happily jump on it rather than deal with this mess. But I can't find anything that will interact happily with the apps, mount network shares of all kinds, and be exceedingly user friendly (take note lin-devs: user-friendly != sys_admin-hell at least it _doesn't have to_). Another way to go about it is to install e.g. ubuntu on a virtual machine and peek under the hood how it works there. But as you say it's probably tied into udev pretty tightly. Tried that too, but each distro has there own "hack" to make it work for them. Crazy huh? How forgiving is devd to a user pulling the plug to early? I did look into it a bit, but it appeared nearly as difficult as deciphering the above scenario- that said, having come through the other side of that I'm not so sure my judgment was very accurate :) So now I might check that fork out and see... Devd just gets some notifications and acts on them. There is a problem with mounted usb devices, but that is one of architecture, I guess. Devd only gets notified _after_ a device has been pulled. There is no way you can prevent data loss in all cases like that. On windows you're supposed to "prepare to eject" a USB device before pulling it out as well. The only "cure" is to mount a device syncronously, and disable _all_ write caching for those devices. If you try that you'll find that doing so has significant performance impact and not in a good way (disks are slw). Almost need a "journaling" system for them. Any thoughts? What about setting up a temp folder (non-volatile buffer?) and a sync? Track devices using the uuid label? God! What a mess... this belongs in the X-files: the truth _is_ out there. But you might lose your head and many years of life just finding the fragments! FreeBSD is on my personal desktop and laptop, but that seems to be the exception rather than the rule. Maybe you should write your experiences up and submit it to the freebsd-doc mailing list for inclusion in the official docs? I may yet do that, but in the interim I'm going to get around to writing up my findings on a lot of different aspects of the systems in a wiki (I'll put up my findings on those as well...). Maybe my pain can help someone else :) For reference _all_ my systems are FreeBSD: from laptops/desktops, HTPC's and servers. I'd like to be able to show a system better and more robust than the alternatives out there as well as easy on the users, and thats what I'm always working towards. And talking about mailing lists, maybe you should try your luck on the freebsd-gnome list? [http://lists.freebsd.org/mailman/listinfo/freebsd-gnome] I would but I'm not subscribed to that one (must be about the only one I'm not on :) ), and it hadn't come to mind as I wasn't using gnome! I'm using nautilus for testing as it has more features, but I'm intending on using pcmanfm or similar- lightweight, but usable. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: PolicyKit confusion
On Sat, Dec 24, 2011 at 01:27:23AM +1000, Da Rock wrote: > couldn't give a shit about network drives. One scenario is network goes > down and I get a screenful of error messages- it annoys the shit out of > me, let alone scaring illiterate users. If the network goes down, network drives won't work. Your users will be sad/scared/frustrated with or without error messages, I'm guessing. > > I don't think enough people care to make it really work under FreeBSD. I've > > certainly never missed it. > They don't care too much under linux either So many years, yet not a > single user doc anywhere on any of the f***ing monsters! Goddamn linux > devs- ever heard of KISS?! How about a user doc for something so bloody > confusing it takes a brain surgeon to figure it out? That's exactly why I've avoided using this stuff. :-) > Ahh, one more thing of note here: polkit-gnome-authorization would not > work under any circumstances (run as root or otherwise) to change > policies! WTF! > > You might take a look at devd(8) as a FreeBSD alternative, but I'm not sure > > if > > it notices new da devices popping up. > > > Oh, believe me I'd happily jump on it rather than deal with this mess. > But I can't find anything that will interact happily with the apps, > mount network shares of all kinds, and be exceedingly user friendly > (take note lin-devs: user-friendly != sys_admin-hell at least it > _doesn't have to_). Another way to go about it is to install e.g. ubuntu on a virtual machine and peek under the hood how it works there. But as you say it's probably tied into udev pretty tightly. > How forgiving is devd to a user pulling the plug to early? I did look > into it a bit, but it appeared nearly as difficult as deciphering the > above scenario- that said, having come through the other side of that > I'm not so sure my judgment was very accurate :) So now I might check > that fork out and see... Devd just gets some notifications and acts on them. There is a problem with mounted usb devices, but that is one of architecture, I guess. Devd only gets notified _after_ a device has been pulled. There is no way you can prevent data loss in all cases like that. On windows you're supposed to "prepare to eject" a USB device before pulling it out as well. The only "cure" is to mount a device syncronously, and disable _all_ write caching for those devices. If you try that you'll find that doing so has significant performance impact and not in a good way (disks are slw). > God! What a mess... this belongs in the X-files: the truth _is_ out > there. But you might lose your head and many years of life just finding > the fragments! FreeBSD is on my personal desktop and laptop, but that seems to be the exception rather than the rule. Maybe you should write your experiences up and submit it to the freebsd-doc mailing list for inclusion in the official docs? And talking about mailing lists, maybe you should try your luck on the freebsd-gnome list? [http://lists.freebsd.org/mailman/listinfo/freebsd-gnome] Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpO5AT7luNuR.pgp Description: PGP signature
Re: PolicyKit confusion
On 12/24/11 00:22, Roland Smith wrote: On Fri, Dec 23, 2011 at 02:18:19PM +1000, Da Rock wrote: This is doing my head in. :-) I'm trying to get my head around hal, dbus, and PolicyKit, and I've made some inroads on the basics, but I cannot get a few things happening. One: I managed to get network:/// smb shares working in say nautilus (not that I've specifically mounted one- not a windows in sight here thank god!), but I was hoping for NFS shares to show up. Why not just use mount_nfs from your login shell's rc file? I'm dealing with mobile devices (laptops, etc), ergo I need an automount solution. I've been using automounter happily for removable devices, but users are picky and I then can't use amd for nfs and automounter couldn't give a shit about network drives. One scenario is network goes down and I get a screenful of error messages- it annoys the shit out of me, let alone scaring illiterate users. I did come across one interesting observation in my search for the truth: its easier to mount a winblows share than an nfs one! On any *nix no less... I also got the usb disk to show up as a 'place' but when I access it I get permissions issues. This is what I'm hung up on. I'm not using policykit or hal, but I did make a separate group predictably called 'usb' to solve permission problems for USB devices. My user-id is a member of that group. I have to following set in /etc/devfs.rules: [my_devs=10] add path 'da*' mode 0660 group usb add path 'msdosfs/*' mode 0660 group usb add path 'usb/*' mode 0660 group usb add path 'ugen*' mode 0660 group usb add path 'tap*' mode 0660 group wheel add path 'pass*' mode 0660 group cdrom add path 'dri/card*' mode 0666 And in /etc/rc.conf I have activated this ruleset: devfs_system_ruleset="my_devs" With this I can access the necessary devices. Already done, but that doesn't help automounting. I checked out /media/hal-* and I see that the mount occurs only as root. How do I change that exactly? I need it showing for operator group. Is hald running under its own user-id? Try the usb group thing and add the hald user to that group. Interesting theory, but it doesn't hold water. More further... I've searched high and low and googled my brains out, but anything remotely related is for linux and udev. I don't think enough people care to make it really work under FreeBSD. I've certainly never missed it. They don't care too much under linux either So many years, yet not a single user doc anywhere on any of the f***ing monsters! Goddamn linux devs- ever heard of KISS?! How about a user doc for something so bloody confusing it takes a brain surgeon to figure it out? Ok. The final cut... I did finally get hal/polkit/dbus to cooperate in the end. But there were more suspects by the end: %.xsession #!/bin/sh exec xhost +localhost & ## test for an existing bus daemon, just to be safe if test -z "$DBUS_SESSION_BUS_ADDRESS" ; then ## if not found, launch a new one eval 'dbus-launch --sh-syntax --exit-with-session' echo "D-Bus per-session daemon address is: $DBUS_SESSION_BUS_ADDRESS" fi exec /usr/local/libexec/polkit-gnome-authentication-agent-1 & exec xscreensaver & exec ck-launch-session icewm-session So... one needs to start a local dbus session, a polkit-auth session, and a ck-launch-session... And in PolicyKit.conf (I'm not entirely happy with the security here), scratch user or group matches and just add the action with result=yes. Now that will work in a single user situation. There is yet another culprit here: gconf. GConf was the shooter on the grassy knoll causing issues with the mount options and permissions. And the crazy part in all this is there is hardly a scratch of info out there on such complex software that are supposedly interacting together in some way that is not exactly documented and yet is in control of the whole bloody system! We should just all revert to winblows at this rate- there's more docs there than for this stuff! An absolute embarrassment to the FOSS community... Ahh, one more thing of note here: polkit-gnome-authorization would not work under any circumstances (run as root or otherwise) to change policies! WTF! You might take a look at devd(8) as a FreeBSD alternative, but I'm not sure if it notices new da devices popping up. Oh, believe me I'd happily jump on it rather than deal with this mess. But I can't find anything that will interact happily with the apps, mount network shares of all kinds, and be exceedingly user friendly (take note lin-devs: user-friendly != sys_admin-hell at least it _doesn't have to_). How forgiving is devd to a user pulling the plug to early? I did look into it a bit, but it appeared nearly as difficult as deciphering the above scenario- that said, having come through the other side of that I'm not so sure my judgment was very accurate :) So now I might check that fork out and see... God! What a mess... this belongs in the X-files: the tru
Re: PolicyKit confusion
On Fri, Dec 23, 2011 at 02:18:19PM +1000, Da Rock wrote: > This is doing my head in. :-) > I'm trying to get my head around hal, dbus, > and PolicyKit, and I've made some inroads on the basics, but I cannot > get a few things happening. > > One: I managed to get network:/// smb shares working in say nautilus > (not that I've specifically mounted one- not a windows in sight here > thank god!), but I was hoping for NFS shares to show up. Why not just use mount_nfs from your login shell's rc file? > I also got the > usb disk to show up as a 'place' but when I access it I get permissions > issues. This is what I'm hung up on. I'm not using policykit or hal, but I did make a separate group predictably called 'usb' to solve permission problems for USB devices. My user-id is a member of that group. I have to following set in /etc/devfs.rules: [my_devs=10] add path 'da*' mode 0660 group usb add path 'msdosfs/*' mode 0660 group usb add path 'usb/*' mode 0660 group usb add path 'ugen*' mode 0660 group usb add path 'tap*' mode 0660 group wheel add path 'pass*' mode 0660 group cdrom add path 'dri/card*' mode 0666 And in /etc/rc.conf I have activated this ruleset: devfs_system_ruleset="my_devs" With this I can access the necessary devices. > I checked out /media/hal-* and I see that the mount occurs only as root. > How do I change that exactly? I need it showing for operator group. Is hald running under its own user-id? Try the usb group thing and add the hald user to that group. > I've > searched high and low and googled my brains out, but anything remotely > related is for linux and udev. I don't think enough people care to make it really work under FreeBSD. I've certainly never missed it. You might take a look at devd(8) as a FreeBSD alternative, but I'm not sure if it notices new da devices popping up. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgp4h0JXeLrqu.pgp Description: PGP signature
Re: PolicyKit confusion - DBus error org.gtk.Private.RemoteVolumeMonitor.Failed: An operation is already pending
On 12/23/11 16:11, Da Rock wrote: On 12/23/11 15:37, Polytropon wrote: On Fri, 23 Dec 2011 14:18:19 +1000, Da Rock wrote: I checked out /media/hal-* and I see that the mount occurs only as root. How do I change that exactly? I need it showing for operator group. I've searched high and low and googled my brains out, but anything remotely related is for linux and udev. I had a feeling you'd be replying to this one :) I think I remember I got it working some time ago (on a 7.1 system), relying on the Gnome HAL FAQ which stated something like this: File: /usr/local/etc/PolicyKit/PolicyKit.conf For "N", use your user name; I think you can also use more than one "match" section if you want to allow access for other users. However, I doubt all this HAL / DBUS / PolicyKit magic is really intended for multi-user purposes. :-) Followed that and done that, didn't work. Although it does say 'result="yes"' not "user". Sorry: that was the freebsd-gnome FAQ. Do you see any way of using "group" instead of "user"? Incidentally, the disk shows up (in case I wasn't being all too clear), as a user I can't access it. And in the hal-tab it shows -u=0 as I said. I hunted down that scenario and it sent me down a very foggy path using hal-fdi's - setting -uid and -u settings. I then checked out /usr/local/share/hal/fdi/policy/ and found in some files (particularly storage) the "key" options are set to "u=" or "uid=". Ok, more data: I believe I'm getting closer. If I set Polkit.conf to match a user rather than a group I fail immediately with permission errors, same as before. However, if I use the group then I get: DBus error org.gtk.Private.RemoteVolumeMonitor.Failed: An operation is already pending dbus-launch gnome-mount --verbose... blah blah (as per instructions for debug output at freebsd-gnome hal faq) simply hangs, like this: dbus-launch gnome-mount --block --no-ui --verbose --hal-udi '/org/freedesktop/Hal/devices/volume_uuid_1061_EF39' gnome-mount 0.8 ** (gnome-mount:8314): DEBUG: Mounting /org/freedesktop/Hal/devices/volume_uuid_1061_EF39 ** (gnome-mount:8314): DEBUG: read default option 'longnames' from gconf strlist key /system/storage/default_options/vfat/mount_options ** (gnome-mount:8314): DEBUG: read default option '-u=' from gconf strlist key /system/storage/default_options/vfat/mount_options ** (gnome-mount:8314): DEBUG: Mounting /org/freedesktop/Hal/devices/volume_uuid_1061_EF39 with mount_point='', fstype='', num_options=2 ** (gnome-mount:8314): DEBUG: option='longnames' ** (gnome-mount:8314): DEBUG: option='-u=1001' Interestingly enough, if I try it with the user instead of group I get the last option there saying -u=0. And /media/.hal-mtab shows that too. In this case I get just .hal-mtab-lock file. Something interesting just happened- after 20mins or so nautilus barfs up a message about being unable to mount the drive: DBus error org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken. So I'm still attempting to assimilate all that to produce something other than a brainfart, and possibly restore balance to the force in that way. Oh, and to preclude any suggestion to this effect, I have set vfs.usermount=1. Note that HAL also has an option of "fixed mount points" to be set at compile time. I think I had set it... No. At least I don't think I set it. I'm also unsure if NFS mounts are "fixed" or "removable" in PK terminology. Nothing is clear on any of this. Which nutcase designed this anyway? Any documentation is vague and unclear, and the software config itself is about as clear as the weather on venus- and just as toxic too ;) Regarding your second question, I can't provide any further information. I just assume it's a means to turn a safe multi-user system into an insecure single-user system, which is what users expect. :-) Dear god! What is this world coming to? And yet they all clamour to use the computer at the same time... thats the scene I see in families all the time, let alone work operations. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: PolicyKit confusion
On 12/23/11 15:37, Polytropon wrote: On Fri, 23 Dec 2011 14:18:19 +1000, Da Rock wrote: I checked out /media/hal-* and I see that the mount occurs only as root. How do I change that exactly? I need it showing for operator group. I've searched high and low and googled my brains out, but anything remotely related is for linux and udev. I had a feeling you'd be replying to this one :) I think I remember I got it working some time ago (on a 7.1 system), relying on the Gnome HAL FAQ which stated something like this: File: /usr/local/etc/PolicyKit/PolicyKit.conf For "N", use your user name; I think you can also use more than one "match" section if you want to allow access for other users. However, I doubt all this HAL / DBUS / PolicyKit magic is really intended for multi-user purposes. :-) Followed that and done that, didn't work. Although it does say 'result="yes"' not "user". Sorry: that was the freebsd-gnome FAQ. Do you see any way of using "group" instead of "user"? Incidentally, the disk shows up (in case I wasn't being all too clear), as a user I can't access it. And in the hal-tab it shows -u=0 as I said. I hunted down that scenario and it sent me down a very foggy path using hal-fdi's - setting -uid and -u settings. I then checked out /usr/local/share/hal/fdi/policy/ and found in some files (particularly storage) the "key" options are set to "u=" or "uid=". So I'm still attempting to assimilate all that to produce something other than a brainfart, and possibly restore balance to the force in that way. Oh, and to preclude any suggestion to this effect, I have set vfs.usermount=1. Note that HAL also has an option of "fixed mount points" to be set at compile time. I think I had set it... No. At least I don't think I set it. I'm also unsure if NFS mounts are "fixed" or "removable" in PK terminology. Nothing is clear on any of this. Which nutcase designed this anyway? Any documentation is vague and unclear, and the software config itself is about as clear as the weather on venus- and just as toxic too ;) Regarding your second question, I can't provide any further information. I just assume it's a means to turn a safe multi-user system into an insecure single-user system, which is what users expect. :-) Dear god! What is this world coming to? And yet they all clamour to use the computer at the same time... thats the scene I see in families all the time, let alone work operations. Thanks ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: PolicyKit confusion
On Fri, 23 Dec 2011 14:18:19 +1000, Da Rock wrote: > I checked out /media/hal-* and I see that the mount occurs only as root. > How do I change that exactly? I need it showing for operator group. I've > searched high and low and googled my brains out, but anything remotely > related is for linux and udev. I think I remember I got it working some time ago (on a 7.1 system), relying on the Gnome HAL FAQ which stated something like this: File: /usr/local/etc/PolicyKit/PolicyKit.conf For "N", use your user name; I think you can also use more than one "match" section if you want to allow access for other users. However, I doubt all this HAL / DBUS / PolicyKit magic is really intended for multi-user purposes. :-) Note that HAL also has an option of "fixed mount points" to be set at compile time. I think I had set it... I'm also unsure if NFS mounts are "fixed" or "removable" in PK terminology. Regarding your second question, I can't provide any further information. I just assume it's a means to turn a safe multi-user system into an insecure single-user system, which is what users expect. :-) -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
PolicyKit confusion
This is doing my head in. I'm trying to get my head around hal, dbus, and PolicyKit, and I've made some inroads on the basics, but I cannot get a few things happening. One: I managed to get network:/// smb shares working in say nautilus (not that I've specifically mounted one- not a windows in sight here thank god!), but I was hoping for NFS shares to show up. I also got the usb disk to show up as a 'place' but when I access it I get permissions issues. This is what I'm hung up on. I checked out /media/hal-* and I see that the mount occurs only as root. How do I change that exactly? I need it showing for operator group. I've searched high and low and googled my brains out, but anything remotely related is for linux and udev. Two: a big thing I've noticed here on google, lists, whatever is a confusion related to the 'define_admin_auth' group. Some are saying it is a security risk to have an average user access this area, and others are just opening it up. I'm with the former, unless I can be convinced otherwise. Can anyone provide some clarity to this issue? What precisely is the capabilties of the 'admin_auth' group? Cheers ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
