Re: Bad file access on the rise
On Sun, Jun 09, 2013 at 04:55:40PM -0400, Doug Ledford wrote: > I would never encourage it in terms of suggesting people try to second > guess the kernel's rules and limitations. However, you can use such a > technique to weed out otherwise known to fail cases, at least in > instances like this where it is a garbage collection operation and > failure to guess properly is obviously far from critical. You can in some circumstances, and not in others. And in the circumstances where you can't, you're going to generate audit records. That's undesirable, and so we should try to find a solution that works under all circumstances, not just the ones where we can add a stat() in front and call it done. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Packaging alien, debhelper and po-debconf
On Dom, 2013-06-09 at 13:02 +0300, Oron Peled wrote: > Some progress... > > > > On Saturday 08 June 2013 10:47:21 Sérgio Basto wrote: > > > > I'm going , if no problem, begin F18 cycle > > > > build this 7 packages: > > > > dpkg > > > > This is what debhelper is waiting for > > > > > > debconf > > > > Went to give it some karma, found an irritating bug. > > Now it is fixed and pushed to rawhide, f19, f18 (waiting for karma > > on the last two) > > > > > > po-debconf > > > > Nothing to build here: po-debconf-0:1.0.16-1.nmu2.fc18.noarch is > > in f18 for quite some time now. > > > > > > debhelper > > > > As soon as dpkg-1.16.x hits f18. > > > > > > alien > > > > dh-make > > > > pbuilder > > > > +1 > > > > > > 1st build dpkg and debconf > > > > > > https://koji.fedoraproject.org/koji/packageinfo?packageID=9963 > > > dpkg-1.16.10-4 for devel , f19 and f18 > > > > Just gave it some karma (f18, f19) > > > > > https://koji.fedoraproject.org/koji/packageinfo?packageID=13893 > > > debconf-1.5.49-2.fc18 for devel , f19 and f18 > > > > Waiting for fixed version (1.5.49-3.fc*) to get pushed to > repositories. > > > > > > after build po-debconf (depends on debconf) > > > > > > https://koji.fedoraproject.org/koji/packageinfo?packageID=16122 > > > we need rebuild po-debconf for devel , f19 and f18 , for new > > > debconf-1.5.49 > > > I put debconf in buildroot-override > > > bodhi --buildroot-override=debconf-1.5.49-2.fc18 --duration=5 > > > --notes="For po-debconf, debhelper, alien and dh-make" > > > > I may be missing something. How do I tell koji to build inside the > buildroot override? yes simple like that :) , say to builder add to buildroot the new package. I just done this command: bodhi --buildroot-override=debconf-1.5.49-3.fc18 --duration=5 --notes="For po-debconf, debhelper, alien and dh-make" -u myuser Response: Your buildroot override has been successfully tagged. It may take up to 20 minutes for the buildroot to regenerate. You can wait for the new buildroot by running `koji wait-repo f18-build --build=debconf-1.5.49-3.fc18` you may see when using: bodhi --list-overrides -u youruser koji wait-repo f18-build --build=debconf-1.5.49-3.fc18 Successfully waited 0:02 for debconf-1.5.49-3.fc18 to appear in the f18-build repo I also add dpkg-1.16.10-4.fc18 to the buildroot right now , so after we know that dpkg-1.16.10-4.fc18 is in buildroot with : koji wait-repo f18-build --build=dpkg-1.16.10-4.fc18 According my notes [1] you may submit debhelper. I don't know if you want rebuild and resubmit po-debconf for updated debconf, I think we should do it first and build debhelper with new po-debconf (on buildroot). [1] 1st build dpkg and debconf after build po-debconf (depends on debconf) after build debhelper ( depends on dpkg and po-debconf ) after build alien (depends on debhelper) and dh-make (depends on debhelper ) after review and build pbuilder (depends on dh-make, debhelper and po-debconf ) Thanks, -- Sérgio M. B. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On 06/09/2013 11:42 AM, Matthew Garrett wrote: > On Sun, Jun 09, 2013 at 11:05:44AM -0400, Doug Ledford wrote: > >> And really, we've spent more time on this thread than it would take >> Lennart to fix PA. Just a quick stat and check of uid before trying to >> remove the stale files and this would all go away. Sure, your stat and >> remove could race, but this is nothing more than a garbage collection >> process anyway, so who cares? We'll just get it next time. > > "Stat and then open" is inherently open to race conditions. Of course it is. > In this > specific case it probably doesn't realistically matter, Exactly my point. > but it's not > behaviour that should be encouraged. I would never encourage it in terms of suggesting people try to second guess the kernel's rules and limitations. However, you can use such a technique to weed out otherwise known to fail cases, at least in instances like this where it is a garbage collection operation and failure to guess properly is obviously far from critical. > The only way to know for certain > whether you're going to be able to open or unlink a file is to attempt > to open or unlink that file, and we shouldn't encourage people to think > otherwise. Agreed. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On 06/09/2013 01:05 PM, Adam Williamson wrote: > On Sun, 2013-06-09 at 16:44 +0200, Reindl Harald wrote: > >> where did you read this? > > Where Doug et al keep not responding to Lennart and Matthew's queries as > to whether the correct fix is to the app or to the audit system, but > keep repeating that 'there's an audit error!' Hey! How did I take top billing in your claim of repeating 'there's an auditing error' without discussing anything about where it should be fixed? A) I didn't really come into the conversation (aside from talking about performance metrics, nothing else) until after you made your claim of Steve repeating the same thing as the Holy Grail of Audit. B) What responses I've made have pretty clearly identified why I thought audit events are bad, and why it is more problematic to create exceptions in the audit subsystem than people originally give credit for...I most certainly have not just repeated some mantra blindly. Please attribute things correctly. >> do yourself a favour and go back to the start of the thread >> there was a problem mentioned which in fact exists >> >> you can discuss how big it is and for whom >> but you could also spend the time to fix the root cause > > Unfortunately I can't; it's not within my realm of authority or > expertise. I was just observing the flow of the discussion and noticing > that it seemed to be somewhat logjammed. I wasn't planning on posting > any further than this, as I think I made the point I meant to... > -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Sun, 2013-06-09 at 16:44 +0200, Reindl Harald wrote: > where did you read this? Where Doug et al keep not responding to Lennart and Matthew's queries as to whether the correct fix is to the app or to the audit system, but keep repeating that 'there's an audit error!' > do yourself a favour and go back to the start of the thread > there was a problem mentioned which in fact exists > > you can discuss how big it is and for whom > but you could also spend the time to fix the root cause Unfortunately I can't; it's not within my realm of authority or expertise. I was just observing the flow of the discussion and noticing that it seemed to be somewhat logjammed. I wasn't planning on posting any further than this, as I think I made the point I meant to... -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Sun, 2013-06-09 at 10:03 -0400, Steve Grubb wrote: > Why would anyone write software that is incorrect enough the OS spits it back > as EINVAL? One example is the btrfs ioctl() for reflink: https://bugzilla.gnome.org/show_bug.cgi?id=626497 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Sun, Jun 09, 2013 at 11:05:44AM -0400, Doug Ledford wrote: > And really, we've spent more time on this thread than it would take > Lennart to fix PA. Just a quick stat and check of uid before trying to > remove the stale files and this would all go away. Sure, your stat and > remove could race, but this is nothing more than a garbage collection > process anyway, so who cares? We'll just get it next time. "Stat and then open" is inherently open to race conditions. In this specific case it probably doesn't realistically matter, but it's not behaviour that should be encouraged. The only way to know for certain whether you're going to be able to open or unlink a file is to attempt to open or unlink that file, and we shouldn't encourage people to think otherwise. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Sun, Jun 09, 2013 at 10:03:19AM -0400, Steve Grubb wrote: > There isn't a mechanism to allow these to slip through. Over the years I have > come to realize that the audit system can be a great resource for debugging > user space. It was sitting through one of Dave Jones' why userspace sucks > lectures and afterwards pouring through audit logs that I saw that we can > find > some of these problems. If part of the goals when writing software is > correctness and efficiency, then wouldn't failing syscalls be of interest? > Not > just in the case of EPERM, but also for example EINVAL? Because this is the expected behaviour in certain cases? I'm not disputing the usefulness of generating these reports, but there are cases where it's entirely legitimate to receive an EPERM and do something useful with that. The audit system needs to recognise that and provide a mechanism for packages to flag that such accesses are genuine and uninteresting. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Sat, 08.06.13 11:35, Adam Williamson (awill...@redhat.com) wrote: > On Sat, 2013-06-08 at 09:25 -0400, Steve Grubb wrote: > > > Its not quite like this. What I need is the OS to be well behaved under > > normal > > conditions so that when problems come along they are easily spotted. Fedora > > has been a fairly well behaved OS over the years. I have had to get a few > > apps > > fixed in the past and the maintainers have always been accommodating. But > > this > > time I am finding we have a serious problem worse than in the past. > > Well, you're defining something as 'bad behaviour' fairly arbitrarily - > or at least controversially: not everyone agrees with your definition. > Continuing to simply assert that the behaviour is bad is not driving the > conversation forward, you're just repeating a position that others have > already raised objections to. Those who are disputing your position are > not saying 'this behaviour is not happening', they are saying 'we > disagree with your definition of "bad behaviour"'. > > If it's not 'bad behaviour', the fact that it didn't happen before is > fairly irrelevant. I could come up with any arbitrary 'test' for some > action that Fedora 19 does that Fedora 18 does not; that doesn't mean I > can then show up on the list waving my test results about and declaring > that there's a problem. First there has to be solid agreement that I'm > actually testing for something we shouldn't be doing. Actually it's worse than that. What Steve proposes as a programming style is something I'd consider actively bad, something that leads to less secure, racy code. And then, I totally don't like this alarmist sound to it, I mean, nothing really changed in PA in the past 5 years, so I really fail to see what this is all about... There isn't any new "trend" I could see here... Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Fri, 07.06.13 22:33, Richard W.M. Jones (rjo...@redhat.com) wrote: > On Fri, Jun 07, 2013 at 06:55:46PM +0200, Lennart Poettering wrote: > > User "simo" creates /dev/shm/1000/ even though 1000 is the UID of user > > "lennart". Lennart can never start PA again, ever. And can't do anything > > about it, because "simo" is in control, and /dev/shm is sticky. > > For /run we create /run/user/ in pam_systemd (I think?). > Can we do the same for /dev/shm/? There's no benefit in doing that. /run/user is not world-writable. Hence creating this dir at login time is totally safe, since only trusted code can create dirs in there. This is different for /dev/shm which is world-writable, and where creating dirs at login doesn't solve anything, because any unprivileged user could easily create dirs for all users and then make it impossible to log in for them. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Fri, 07.06.13 20:21, Miloslav Trmač (m...@volny.cz) wrote: > AFAICS this can be solved by a ~100-line? patch to the pulseaudio > ecosystem, or by doing much more work and coordicnation to design a > different shm mechanism or a new shm namespace. I'm not planning to > work on it so I suppose my opinion has little weight, but this doesn't > seem like a difficult decision at all. This is not that easy. In the PA protocol we pass around triplets of numeric SHM segment IDs, offsets into the SHM segment plus a length. That's all there is. If we cannot access one of these segments, that's something the kernel should tell us, and we shouldn't have to pass any additional (and most likely incorrect) meta data around about that... Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Fri, 07.06.13 14:39, Bill Nottingham (nott...@redhat.com) wrote: > Lennart Poettering (mzerq...@0pointer.de) said: > > Yes, it is. > > > > POSIX shared memory doesn't define any useful scheme for automatic > > removing of shared memory segments from /dev/shm after use. Hence, in > > order to make sure that left-over segments don't fill up /dev/shm > > forever PA will try to GC dead segments from /dev/shm on each > > start-up. For that it iterates through /dev/shm/pulse-shm*, tries to > > read the PID that is stored in there. When the PID still exists it goes > > to the next file. If the PID doesn't exist it unlinks the file and then > > goes to the next one. It's a simple scheme that works around the > > limitations of POSIX shm. Of course /dev/shm is a single namespace for > > all users, hence not all files can be opened, and that's totally cool > > and expected, and they will be skipped. > > > > Shared memory on Linux is a mess. Not automatic clean up, no quota > > limits, it's a sad story. If you care about security and reliability, it > > would be great doing something about this, so that arbitrary users > > cannot DoS the system this easily anymore... > > Any reason we don't run with namespaced /dev/shm vis-a-vis private /tmp? We kinda have that in XDG_RUNTIME_DIR now. /dev/shm is supposed to be shared among users I guess. If you want a user-specific namespace use XDG_RUNTIME_DIR instead, you will get pretty much the same semantics except that shm_open() needs to be replaced by open(). Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Fri, 07.06.13 15:35, Steve Grubb (sgr...@redhat.com) wrote: > On Friday, June 07, 2013 07:29:56 PM Matthew Garrett wrote: > > On Fri, Jun 07, 2013 at 02:02:14PM -0400, Simo Sorce wrote: > > > The point is that we are simply throwing ideas off the wall as an aid in > > > finding a way to solve the issue for all. > > > > So why not add a mechanism to permit applications to indicate that > > certain accesses they make should be ignored by audit? > > We've never needed an exception in the past. What I'm reporting is there is > now a trend on the rise where apps are trying to open files that do not > belong > to them or open them not wanting the access time updated which attempts to > bypass forensic time stamps. This is hardly a "new trend" btw. PA has been doing this since about forever and has been default since Fedora 8. Which is more than 5 years ago. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On 06/09/2013 10:34 AM, Adam Williamson wrote: > On Sun, 2013-06-09 at 10:03 -0400, Steve Grubb wrote: > >>> I don't think anyone wants these accesses to generate audit records. The >>> question is whether the right way to fix that is to avoid those accesses >>> in the first place or to provide a mechanism so that legitimate accesses >>> don't generate audit records. >> >> There isn't a mechanism to allow these to slip through. Over the years I >> have >> come to realize that the audit system can be a great resource for debugging >> user space. It was sitting through one of Dave Jones' why userspace sucks >> lectures and afterwards pouring through audit logs that I saw that we can >> find >> some of these problems. If part of the goals when writing software is >> correctness and efficiency, then wouldn't failing syscalls be of interest? >> Not >> just in the case of EPERM, but also for example EINVAL? > > Well what I'm trying to say is that you're acting as if the entire > 'audit system' was carved on stone tablets and handed down from God. It > wasn't. It's just a set of checks, the logic behind _each of which_ is > as open to question as anything else. Just because a test for all EPERM > syscall fails is a part of 'the audit system' does not make it an > unquestionable totem. Instead of answering the question "do we actually > believe that all cases of EPERM should be 'fixed', or in some cases > would the cure be worse than the disease?" you seem to just keep saying > "The Holy Audit System told me there's a problem!" > > I don't know who's right, in this case. But looking at the debate, I see > one side raising what looks like a legitimate line of inquiry, and you > just batting it back with 'The Holy Audit System has no flaws'. > > "There isn't a mechanism", okay, point taken. But that can be a flaw of > the audit system as much as anything else. Not necessarily. The audit system is part of a security verification system. Those have always had different rules than, as you bring up later, a test suite. I remember the days when it was considered safe practice to plug a dot-matrix line printer into your server and pipe the output of syslog directly to the line printer so that if you got hacked, the hackers could not erase their log trail. The audit system is just a more modern version of that same thing. And the second you put any sort of exception into the audit rules, then you have to verify that the exception can never be used to circumvent the legitimate purpose of the rule. So, if we put an exception into the system so that PulseAudio can open these shm files and not be audited, we would have to prove conclusively that no other application can ever use that exception to hide their tracks (a rather difficult task) or risk loosing some of our security certifications. At least, that's my understanding of how these security certifications work, Steve can verify this. So is the audit system the "Holy Audit System"? Not in so many words, but in some ways, yes, it is. It is not the same as the rest of the system. Fedora has its concept of critical path packages for the release, and for security certifications there is an entirely different critical path set and the audit subsystem is right up top on that list, and the kernel is right up there with it. So if your first thought about any conflict between the audit subsystem and normal packages is "Should we fix this in the audit subsystem or in the package?", then a necessary part of evaluating whether or not it should be fixed in the audit subsystem *must* be adding the burden of proving that your fix in the audit subsystem can not be abused by malicious hackers to help subvert your system (or to hide the evidence of their subversion/attempts). If you aren't immediately considering that burden when thinking about where to fix things, then you aren't considering the entirety of the work that must be done to solve the problem in the audit space. >> Why would anyone write software that is incorrect enough the OS spits it >> back >> as EINVAL? > > This is entirely irrelevant. From a QA monkey perspective, I'm comparing > this with the case where we have a suite of tests, and someone raises > the question if one of them is a sensible test. Talking about how good > one of the others is is entirely out of scope. The fact we put them all > together and called them a 'test suite' is really neither here nor > there. The question here is not 'is auditing useful?', it's 'is this > particular audit check one which always indicates a genuine bug that > must be fixed?' > >> I'll leave it here for anyone curious enough to dig out the details of how >> each syscall is wrong. But its my belief that these are not intentionally >> written to fail and people didn't know they were issuing syscalls that will >> never work. > > Well, that's clearly not the case in the situation we're actually > discussing: the author of one of the pieces of soft
Re: Bad file access on the rise
On Fri, 07.06.13 14:02, Simo Sorce (s...@redhat.com) wrote: > > Well, you know, this problem isn't new. Some SELinux AVCs can be set to > > ignored for precisely reasons like this one, because it is common that > > things like these happen: accesses which fail where that is > > expected. You can call me a troll as much as you want, but then please > > call the Selinux folks trolls first. > > The SELinux folks, being the practical bunch they are simply work around > bad programs. That doesn't mean they are trolls. > Also the SeLinux folks have 'context', which is something an audit > system may not have. > Finally the fact SELinux folks are considerate does not mean that > programs need not be fixed to behave better. > > Of course the author can always decide to show the finger and do what he > wants anyway, we live in a free and unpolite world after all. Well, you know this is not about me showing anybody the finger. It's just that I fundamentally believe that the programming practices the audit folks propose are systematically wrong, and lead to worse code. You should not second guess the kernel, and try to bypass the kernel's access checks by doing them in userspace. You cannot win that game, because you simply don't have the same information the kernel has. The kernel is both more restrictive and less restrictive than simple UID checks they way they have been proposed on this thread, so you have the best chance to be wrong with manual userspace UID checks. More restrictive are SELinux checks, less restrictive are additional ACLs that might be on a file. And to mix things up entirely with capabilities everything becomes extra confusing. Then, doing these things in userspace makes these checks non-atomic. If the kernel does these checks, then the access checks and the open() will be atomic. Of course, one can deal with these atomicity problems... or one can just do it properly and leave all this to the kernel... So yeah, by doing these checks in userspace they way you suggest: a) adds more code to userspace b) second guesses the kernel, a game which you cannot win c) creates atomicity problems So, yeah, just trying to open the shm files *is the right thing to do*. And if audit doesn't like that, then audit needs to be fixed, not PA. > > User "simo" creates /dev/shm/1000/ even though 1000 is the UID of user > > "lennart". Lennart can never start PA again, ever. And can't do anything > > about it, because "simo" is in control, and /dev/shm is sticky. > > That could be solved by precreating the directory at login time, like we > do for many other things. There isn't just one way. > Or you could even stop using /dev/shm and we'd have one less problem > entirely :) Login time doesn't suffice. Isn't that trivial to see? "Simo" logs in first, his directory is created. Then he creates also /dev/shm/1000, which is still lennart's UID. Now Lennart logs in, and bam, now it's even worse, he cannot even log in anymore since his directory already exists. So, the only option you have is to create these dirs during boot-up before the first user can log in. But in a world of LDAP that's not really too useful, is it? > The point is that we are simply throwing ideas off the wall as an aid in > finding a way to solve the issue for all. Quite honestly, these problems are well known. World-writable directories with a shared namespace are inherently broken, you can only create randomized files/dirs in them, everything else is necessarily broken. > > It's really disappointing if people as seasoned as you still don't > > understand these shared-namespace issues of world-writable directories. > > Yeah, just like it is disappointing to see that your first reaction is > always to show the finger hard and solid instead of trying to have a > reasonable discussion about the problem. The problem of shared namespace world-writable dirs is hardly new, is it? Anyway, this is not constructive. Let's stop this here. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On 06/09/2013 09:53 AM, Roberto Ragusa wrote: > On 06/08/2013 04:13 PM, Doug Ledford wrote: >> >> Yes, but none of these results show the .12s time that your first >> noatime test run showed in your original post. If you are now saying >> that atime is faster than noatime by about .005 to .010s, then these >> results seem to show that. But your original post was from .019 to .12, >> or a difference of .10+s. That was cache load time, not just the >> syscall difference. > > Hmm, someone is misreading the results. > I've reread multiple times, and I see a difference of 12s, not .12s. > > ---> real 0m12.645s > ---> user 0m0.003s > ---> sys 0m0.159s > > And 12 seconds (elapsed, with 0.159s system) means 12s/5000=2.4ms > which could only be explained with the auditing system doing fsync > calls on its log files. > You're right, I was going from memory and I did not remember it being 12 seconds, I thought it was .12 seconds. This most certainly is not just the effects of loading cache. A 12 second cache load would only happen if you are loading from stone tablet... -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Sun, 2013-06-09 at 10:03 -0400, Steve Grubb wrote: > > I don't think anyone wants these accesses to generate audit records. The > > question is whether the right way to fix that is to avoid those accesses > > in the first place or to provide a mechanism so that legitimate accesses > > don't generate audit records. > > There isn't a mechanism to allow these to slip through. Over the years I have > come to realize that the audit system can be a great resource for debugging > user space. It was sitting through one of Dave Jones' why userspace sucks > lectures and afterwards pouring through audit logs that I saw that we can > find > some of these problems. If part of the goals when writing software is > correctness and efficiency, then wouldn't failing syscalls be of interest? > Not > just in the case of EPERM, but also for example EINVAL? Well what I'm trying to say is that you're acting as if the entire 'audit system' was carved on stone tablets and handed down from God. It wasn't. It's just a set of checks, the logic behind _each of which_ is as open to question as anything else. Just because a test for all EPERM syscall fails is a part of 'the audit system' does not make it an unquestionable totem. Instead of answering the question "do we actually believe that all cases of EPERM should be 'fixed', or in some cases would the cure be worse than the disease?" you seem to just keep saying "The Holy Audit System told me there's a problem!" I don't know who's right, in this case. But looking at the debate, I see one side raising what looks like a legitimate line of inquiry, and you just batting it back with 'The Holy Audit System has no flaws'. "There isn't a mechanism", okay, point taken. But that can be a flaw of the audit system as much as anything else. > Why would anyone write software that is incorrect enough the OS spits it back > as EINVAL? This is entirely irrelevant. From a QA monkey perspective, I'm comparing this with the case where we have a suite of tests, and someone raises the question if one of them is a sensible test. Talking about how good one of the others is is entirely out of scope. The fact we put them all together and called them a 'test suite' is really neither here nor there. The question here is not 'is auditing useful?', it's 'is this particular audit check one which always indicates a genuine bug that must be fixed?' > I'll leave it here for anyone curious enough to dig out the details of how > each syscall is wrong. But its my belief that these are not intentionally > written to fail and people didn't know they were issuing syscalls that will > never work. Well, that's clearly not the case in the situation we're actually discussing: the author of one of the pieces of software you audited says he knows about the failed syscalls and does not think they're a problem. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On Sunday, June 09, 2013 05:56:42 AM Matthew Garrett wrote: > On Sat, Jun 08, 2013 at 08:28:48PM -0400, Doug Ledford wrote: > > On 06/08/2013 02:35 PM, Adam Williamson wrote: > > > Well, you're defining something as 'bad behaviour' fairly arbitrarily - > > > or at least controversially: not everyone agrees with your definition. > > > > Speaking as a former sysadmin responsible for intrusion detection, this > > is not a controversial definition at all (namely that anything that > > creates audit events without a reasonably just cause is 'bad behavior'). > > It is the only sane definition of 'bad behavior'. Anything that makes > > an admin go chasing ghosts for no good reason is most definitely 'bad > > behavior', and every single audit event on a system must be identifiable > > by the admins before you know your system is secure. Thanks Doug. I don't think I could have said it better myself. These events also consume space. Each one of these occupy about 600 bytes. But when you multiply that by the number I gave you yesterday for a week of these events, it gets large. Because I work on the audit system, I have a 100Mb dedicated partition for audit logs. Until recently, that would hold about a year's worth of events. The audit rules have not changed in maybe 4 years. When I run aureport --log today, my logs only go back to May 30 of this year. That's how big of an impact this is. As to the reason why I didn't bring this up earlier, upstream broke the audit system on the 3.7 kernel and it wasn't fixed until the 3.9.4 kernel. > I don't think anyone wants these accesses to generate audit records. The > question is whether the right way to fix that is to avoid those accesses > in the first place or to provide a mechanism so that legitimate accesses > don't generate audit records. There isn't a mechanism to allow these to slip through. Over the years I have come to realize that the audit system can be a great resource for debugging user space. It was sitting through one of Dave Jones' why userspace sucks lectures and afterwards pouring through audit logs that I saw that we can find some of these problems. If part of the goals when writing software is correctness and efficiency, then wouldn't failing syscalls be of interest? Not just in the case of EPERM, but also for example EINVAL? Why would anyone write software that is incorrect enough the OS spits it back as EINVAL? I think its because they didn't know and in some cases it was written on another OS that accepts that particular input. To that point, you can load these two rules: auditctl -a exit,never -F arch=b64 -S rt_sigreturn auditctl -a exit,always -S all -F exit=-EINVAL -k einval-test and then use the computer for a while. To see what you get: # ausearch --start today -k einval -m SYSCALL --raw | aureport -x --summary Executable Summary Report = total file = 165 /usr/libexec/mysqld 149 /usr/bin/man 105 /usr/lib/udisks2/udisksd 74 /usr/sbin/libvirtd 16 /usr/lib/systemd/systemd-udevd 8 /usr/lib64/firefox/firefox 5 /usr/bin/python2.7 2 /usr/sbin/modem-manager 2 /usr/sbin/NetworkManager 2 /usr/bin/gnome-terminal 1 /usr/bin/qemu-kvm 1 /usr/bin/knotify4 1 /usr/bin/xchat Who knew all these programs were making invalid syscalls? Which syscalls are we having a problem with? # ausearch --start today -k einval -m SYSCALL --raw | aureport -i --syscall -- summary Syscall Summary Report == total syscall == 320 readlink 149 munmap 74 rt_sigaction 9 ioctl I'll leave it here for anyone curious enough to dig out the details of how each syscall is wrong. But its my belief that these are not intentionally written to fail and people didn't know they were issuing syscalls that will never work. The audit system can be used to find software problems so the system operates more efficiently. Thanks, -Steve -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Bad file access on the rise
On 06/08/2013 04:13 PM, Doug Ledford wrote: > > Yes, but none of these results show the .12s time that your first > noatime test run showed in your original post. If you are now saying > that atime is faster than noatime by about .005 to .010s, then these > results seem to show that. But your original post was from .019 to .12, > or a difference of .10+s. That was cache load time, not just the > syscall difference. Hmm, someone is misreading the results. I've reread multiple times, and I see a difference of 12s, not .12s. ---> real 0m12.645s ---> user 0m0.003s ---> sys0m0.159s And 12 seconds (elapsed, with 0.159s system) means 12s/5000=2.4ms which could only be explained with the auditing system doing fsync calls on its log files. -- Roberto Ragusamail at robertoragusa.it -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
F-19 Branched report: 20130609 changes
Compose started at Sun Jun 9 09:15:02 UTC 2013 Broken deps for x86_64 -- [deltacloud-core] deltacloud-core-rhevm-1.1.3-1.fc19.noarch requires rubygem(rbovirt) >= 0:0.0.18 [dragonegg] dragonegg-3.1-19.fc19.x86_64 requires gcc = 0:4.7.2-9.fc19 [freeipa] freeipa-server-strict-3.2.0-2.fc19.x86_64 requires krb5-server = 0:1.11.2 [gooddata-cl] gooddata-cl-1.2.56-2.fc19.noarch requires gdata-java [kawa] 1:kawa-1.11-5.fc19.x86_64 requires servlet25 [koji] koji-vm-1.8.0-1.fc19.noarch requires python-virtinst [libkolab] php-kolab-0.4.1-3.fc19.x86_64 requires php(zend-abi) = 0:20100525-x86-64 php-kolab-0.4.1-3.fc19.x86_64 requires php(api) = 0:20100412-x86-64 [llvm] clang-3.3-0.4.rc2.fc19.i686 requires libstdc++-devel = 0:4.8.0 clang-3.3-0.4.rc2.fc19.i686 requires gcc = 0:4.8.0 clang-3.3-0.4.rc2.fc19.x86_64 requires libstdc++-devel = 0:4.8.0 clang-3.3-0.4.rc2.fc19.x86_64 requires gcc = 0:4.8.0 [nodejs-tilelive] nodejs-tilelive-4.4.3-2.fc19.noarch requires npm(optimist) < 0:0.4 [openbox] gdm-control-3.5.0-11.20121001git782b28.fc19.x86_64 requires gnome-panel gnome-panel-control-3.5.0-11.20121001git782b28.fc19.x86_64 requires gnome-panel [ovirt-engine] ovirt-engine-notification-service-3.1.0-1.fc19.noarch requires classpathx-mail [ovirt-guest-agent] ovirt-guest-agent-gdm-plugin-1.0.6-6.fc19.x86_64 requires libgdmsimplegreeter.so.1()(64bit) [perl-Bio-ASN1-EntrezGene] perl-Bio-ASN1-EntrezGene-1.091-17.fc19.noarch requires perl(Bio::Index::AbstractSeq) [perl-Bio-SamTools] perl-Bio-SamTools-1.35-2.fc19.x86_64 requires perl(Bio::SeqFeature::Lite) perl-Bio-SamTools-1.35-2.fc19.x86_64 requires perl(Bio::PrimarySeq) [python-TraitsBackendQt] python-TraitsBackendQt-3.5.0-5.fc19.noarch requires python-TraitsGUI [scala] scala-2.9.2-2.fc19.noarch requires osgi(org.scala-ide.scala.library) [spacewalk-web] spacewalk-dobby-1.9.22-2.fc19.noarch requires perl(Spacewalk::Setup) [zarafa] php-mapi-7.0.13-1.fc19.x86_64 requires php(zend-abi) = 0:20100525-x86-64 php-mapi-7.0.13-1.fc19.x86_64 requires php(api) = 0:20100412-x86-64 Broken deps for i386 -- [deltacloud-core] deltacloud-core-rhevm-1.1.3-1.fc19.noarch requires rubygem(rbovirt) >= 0:0.0.18 [dragonegg] dragonegg-3.1-19.fc19.i686 requires gcc = 0:4.7.2-9.fc19 [freeipa] freeipa-server-strict-3.2.0-2.fc19.i686 requires krb5-server = 0:1.11.2 [gooddata-cl] gooddata-cl-1.2.56-2.fc19.noarch requires gdata-java [kawa] 1:kawa-1.11-5.fc19.i686 requires servlet25 [koji] koji-vm-1.8.0-1.fc19.noarch requires python-virtinst [libkolab] php-kolab-0.4.1-3.fc19.i686 requires php(zend-abi) = 0:20100525-x86-32 php-kolab-0.4.1-3.fc19.i686 requires php(api) = 0:20100412-x86-32 [llvm] clang-3.3-0.4.rc2.fc19.i686 requires libstdc++-devel = 0:4.8.0 clang-3.3-0.4.rc2.fc19.i686 requires gcc = 0:4.8.0 [nodejs-tilelive] nodejs-tilelive-4.4.3-2.fc19.noarch requires npm(optimist) < 0:0.4 [openbox] gdm-control-3.5.0-11.20121001git782b28.fc19.i686 requires gnome-panel gnome-panel-control-3.5.0-11.20121001git782b28.fc19.i686 requires gnome-panel [ovirt-engine] ovirt-engine-notification-service-3.1.0-1.fc19.noarch requires classpathx-mail [ovirt-guest-agent] ovirt-guest-agent-gdm-plugin-1.0.6-6.fc19.i686 requires libgdmsimplegreeter.so.1 [perl-Bio-ASN1-EntrezGene] perl-Bio-ASN1-EntrezGene-1.091-17.fc19.noarch requires perl(Bio::Index::AbstractSeq) [perl-Bio-SamTools] perl-Bio-SamTools-1.35-2.fc19.i686 requires perl(Bio::SeqFeature::Lite) perl-Bio-SamTools-1.35-2.fc19.i686 requires perl(Bio::PrimarySeq) [python-TraitsBackendQt] python-TraitsBackendQt-3.5.0-5.fc19.noarch requires python-TraitsGUI [scala] scala-2.9.2-2.fc19.noarch requires osgi(org.scala-ide.scala.library) [spacewalk-web] spacewalk-dobby-1.9.22-2.fc19.noarch requires perl(Spacewalk::Setup) [zarafa] php-mapi-7.0.13-1.fc19.i686 requires php(zend-abi) = 0:20100525-x86-32 php-mapi-7.0.13-1.fc19.i686 requires php(api) = 0:20100412-x86-32 New package: goaccess-0.5-2.fc19 Apache Log Analyzer New package: reposurgeon-2.39-2.fc19 SCM Repository Manipulation Tool New package: wemux-2.2.0-1.fc19 A tool help improve multi-user terminal multiplexing New package: zimlib-1.0-5.fc19 Library for reading/writing ZIM files Updated Packages: ModemManager-0.6.2.0-1.fc19 --- * Fri May 31 2013 Dan Williams - 0.6.2.0-1 - Update to 0.6.2 final release - Fix crash disabling CDMA modems - Fix PDP context matching when IPV6 contexts are defined * Tue May 28 2013 Dan Williams - 0.6.0.0-4 - U
rawhide report: 20130609 changes
Compose started at Sun Jun 9 08:15:02 UTC 2013 Broken deps for x86_64 -- [bind10] bind10-1.0.0-3.fc20.i686 requires liblog4cplus-1.1.so.5 bind10-1.0.0-3.fc20.x86_64 requires liblog4cplus-1.1.so.5()(64bit) bind10-dhcp-1.0.0-3.fc20.i686 requires liblog4cplus-1.1.so.5 bind10-dhcp-1.0.0-3.fc20.x86_64 requires liblog4cplus-1.1.so.5()(64bit) bind10-dns-1.0.0-3.fc20.i686 requires liblog4cplus-1.1.so.5 bind10-dns-1.0.0-3.fc20.x86_64 requires liblog4cplus-1.1.so.5()(64bit) [ekiga] ekiga-4.0.1-1.fc19.x86_64 requires libedata-book-1.2.so.17()(64bit) [gdb-heap] gdb-heap-0.5-12.fc19.x86_64 requires glibc(x86-64) = 0:2.17 [gooddata-cl] gooddata-cl-1.2.56-2.fc19.noarch requires gdata-java [kawa] 1:kawa-1.11-5.fc19.x86_64 requires servlet25 [koji] koji-vm-1.8.0-1.fc20.noarch requires python-virtinst [lancet] lancet-1.0.1-6.fc19.noarch requires ant-nodeps >= 0:1.7.1 [lutok] lutok-devel-0.2-4.fc19.i686 requires lua-devel < 0:5.2 lutok-devel-0.2-4.fc19.x86_64 requires lua-devel < 0:5.2 [openbox] gdm-control-3.5.0-11.20121001git782b28.fc19.x86_64 requires gnome-panel gnome-panel-control-3.5.0-11.20121001git782b28.fc19.x86_64 requires gnome-panel [ovirt-engine] ovirt-engine-notification-service-3.1.0-1.fc19.noarch requires classpathx-mail [ovirt-guest-agent] ovirt-guest-agent-gdm-plugin-1.0.6-6.fc19.x86_64 requires libgdmsimplegreeter.so.1()(64bit) [oyranos] oyranos-libs-0.4.0-7.fc19.i686 requires libraw.so.5 oyranos-libs-0.4.0-7.fc19.x86_64 requires libraw.so.5()(64bit) [perl-Bio-ASN1-EntrezGene] perl-Bio-ASN1-EntrezGene-1.091-17.fc19.noarch requires perl(Bio::Index::AbstractSeq) [perl-Bio-SamTools] perl-Bio-SamTools-1.35-2.fc19.x86_64 requires perl(Bio::SeqFeature::Lite) perl-Bio-SamTools-1.35-2.fc19.x86_64 requires perl(Bio::PrimarySeq) [python-TraitsBackendQt] python-TraitsBackendQt-3.5.0-5.fc19.noarch requires python-TraitsGUI [python-flask-admin] python-flask-admin-1.0.5-3.fc20.noarch requires python-wtf-peewee [qpid-cpp] qpid-cpp-server-xml-0.20-6.fc20.x86_64 requires libxqilla.so.5()(64bit) [scala] scala-2.9.2-2.fc19.noarch requires osgi(org.scala-ide.scala.library) [spacewalk-web] spacewalk-dobby-1.9.22-2.fc19.noarch requires perl(Spacewalk::Setup) [spring] spring-94.1-1.fc20.x86_64 requires libassimp.so.2()(64bit) [tex-simplecv] tex-simplecv-doc-1.6-12.fc19.noarch requires texlive-texmf-doc [tuna] tuna-0.11-1.fc20.noarch requires python-linux-procfs >= 0:0.4.6 [zarafa] libmapi-7.0.13-1.fc19.i686 requires libicalss.so.0 libmapi-7.0.13-1.fc19.i686 requires libical.so.0 libmapi-7.0.13-1.fc19.x86_64 requires libicalss.so.0()(64bit) libmapi-7.0.13-1.fc19.x86_64 requires libical.so.0()(64bit) php-mapi-7.0.13-1.fc19.x86_64 requires php(zend-abi) = 0:20100525-x86-64 php-mapi-7.0.13-1.fc19.x86_64 requires php(api) = 0:20100412-x86-64 zarafa-ical-7.0.13-1.fc19.x86_64 requires libicalss.so.0()(64bit) zarafa-ical-7.0.13-1.fc19.x86_64 requires libical.so.0()(64bit) Broken deps for i386 -- [bind10] bind10-1.0.0-3.fc20.i686 requires liblog4cplus-1.1.so.5 bind10-dhcp-1.0.0-3.fc20.i686 requires liblog4cplus-1.1.so.5 bind10-dns-1.0.0-3.fc20.i686 requires liblog4cplus-1.1.so.5 [ekiga] ekiga-4.0.1-1.fc19.i686 requires libedata-book-1.2.so.17 [gdb-heap] gdb-heap-0.5-12.fc19.i686 requires glibc(x86-32) = 0:2.17 [gooddata-cl] gooddata-cl-1.2.56-2.fc19.noarch requires gdata-java [kawa] 1:kawa-1.11-5.fc19.i686 requires servlet25 [koji] koji-vm-1.8.0-1.fc20.noarch requires python-virtinst [lancet] lancet-1.0.1-6.fc19.noarch requires ant-nodeps >= 0:1.7.1 [lutok] lutok-devel-0.2-4.fc19.i686 requires lua-devel < 0:5.2 [openbox] gdm-control-3.5.0-11.20121001git782b28.fc19.i686 requires gnome-panel gnome-panel-control-3.5.0-11.20121001git782b28.fc19.i686 requires gnome-panel [ovirt-engine] ovirt-engine-notification-service-3.1.0-1.fc19.noarch requires classpathx-mail [ovirt-guest-agent] ovirt-guest-agent-gdm-plugin-1.0.6-6.fc19.i686 requires libgdmsimplegreeter.so.1 [oyranos] oyranos-libs-0.4.0-7.fc19.i686 requires libraw.so.5 [perl-Bio-ASN1-EntrezGene] perl-Bio-ASN1-EntrezGene-1.091-17.fc19.noarch requires perl(Bio::Index::AbstractSeq) [perl-Bio-SamTools] perl-Bio-SamTools-1.35-2.fc19.i686 requires perl(Bio::SeqFeature::Lite) perl-Bio-SamTools-1.35-2.fc19.i686 requires perl(Bio::PrimarySeq) [python-TraitsBackendQt] python-TraitsBackendQt-3.5.0-5.fc19.noarch requires python-TraitsGUI [python-flask-admin] python-flask-admin-1.0.5-3.fc20.noarch requires python-w
