subject:"Drawing lessons from fatal SELinux bug #1054350"

On Sat, 2014-01-25 at 17:28 -0700, Chris Murphy wrote:
 On Jan 25, 2014, at 12:55 PM, Simo Sorce s...@redhat.com wrote:
 
  The reason is simple: lot's of software *changes* data as part of its
  normal functioning, including and often in rollback-incompatible ways.
  
  You cannot assume that upgrading a program that uses a database X from
  version A to B can still work if you keep database X unchanged and then
  rollback from B to A. Lot of applications apply changes to database at
  upgrade time, either in the rpm scriplets or automatically as soon as a
  new version binary is run.
 
 I wouldn't ever expect this with a minor version or security update.
 I'd consider it a complete betrayal of software versioning to do this.
 In fact in certain instances of major version changes, downward
 compatibility of the file format is expected.

And who ever said 'minor' version ?

However I've done this with minor versions too with internal databases.
There is no betrayal whatsoever, major versions are introduced when you
make user-visible changes or you break an API/ABI, you do not
necessarily change major version for some internal change.

  It is basically impossible to find applications that handle the case
  where you downgrade, in any more graceful way than punting and
 failing
  to start in the *good* case. In the bad case they start and trash
 the
  database.
 
 I guess I'm not really following. Do you have a for instance?

At least 3 or 4 applications I am involved with did this kind of
internal changes.

  Because off hand this sounds like a kind of sabotage to me.

No, it is just normal, everyday, software development.

  If it's throw away database info that can simply be reconstructed I'd
 probably tolerate it. But for that matter, such things should go in an
 defined cache location so that it's not even being backed up.

In some cases it is data that can be reconstructed, so all you need to
do is to manually blow away the files and let the app rebuild them. In
some cases that also have additional inconveniences. In some cases it is
not data you can or want to throw away.

 But important user data having it's format updated in a way that makes
 it incompatible with the previous minor version (same major version)?

So ?
It is only visible if you downgrade which a lot of software do not
support and explicitly so.

  I'm snickering at the language that would ensue in the proprietary
 software world, if I'm not totally confused about what it is you're
 suggested is fair game. It'd be the sort of language you wouldn't want
 your teenager or grandmother to hear.

??

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

On Sat, 2014-01-25 at 17:54 -0700, Chris Murphy wrote:
 On Jan 25, 2014, at 4:12 PM, Adam Williamson awill...@redhat.com wrote:
  
  * Do an offline update that includes Foo v2.0
  * Boot the updated system, run Foo, it migrates its configuration to
  some new scheme
  * Realize there was something wrong with the update, roll it back
  * Run Foo again, find it doesn't work because it's been migrated to the
  new config scheme which the old version of Foo doesn't work with
 
 I would grumble, but a configuration file being updated and made
 incompatible with the prior version would be tolerated. Ideally the
 application makes an unmodified copy. If it doesn't, new school
 restore with --reflink from snapshot, regular cp if using LVM thinp
 snapshots, and old school just restore the file from a conventional
 backup. Not such a big deal.
 
 If it's something far less throw away than configuration files being
 changed, it's a bit more complicated how badly and quickly the
 conversation degrades. But I can hardly recall a recent example of
 this happening. It's just not that common in my experience.

What about mail application change the format of the mail folders ?

It happens, and it is *not* data you want to risk throwing away. There
are many other examples like this especially on the server side.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

On Sat, 2014-01-25 at 15:04 -0500, Colin Walters wrote:
Hi Simo,

On Sat, 2014-01-25 at 14:55 -0500, Simo Sorce wrote:

The reason is simple: lot's of software *changes* data as part of its
normal functioning, including and often in rollback-incompatible ways.

I wrote and maintain a system that has been doing continuous deployment
of GNOME. It's been running for over a year, and is nearing it's
1th build.

I have rolled back many times - both on the server side, and on the
client side. Here's one I *just did* a few minutes ago because vala git
master broke the build of gnome-calculator:

https://git.gnome.org/browse/gnome-continuous/commit/?id=32a52e53100e92aad5d2dfae969be82227322f49

That's me telling the system please stop building git master, and
freeze to this specific commit. All clients get that change when they
upgrade - OSTree cares not at all for version numbers.

The vala maintainers continue to work out the issue in git master, and I
continue to ship a working system. Double win.

Now it's true, programs in GNOME do sometimes make the type of data
format transition you're talking about. Evolution has done it at least
twice.

But you know what? My real world experience has been that having the
ability to roll back has *far* *far* *far* outweighed the downsides when
applications do format transitions. It's comparatively rare.

Far more people are bit by things like hardware-specific issues where
gnome-shell fails to render on this particular card - and rollback works
beautifully for that.

I never said it won't work in absolute, it probably will work ok in many
cases, just to cause incredible issues in others.

It is a fine tool in the hands of an expert that knows how to check
whether reverting to a snapshot is safe.

It is not going to be a good solution for non-expert users though
*unless* you provide system APIs that *all* applications use to signal
when they are doing irreversible changes so that the user can be warned
about potential data loss right when he asks the system to revert a
snapshot.

Simo.

--
Simo Sorce * Red Hat, Inc * New York

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

Am 26.01.2014 20:56, schrieb Chris Murphy:
 What about mail application change the format of the mail folders ?
 
 Good question because I experienced this recently. So the way Apple does this 
 on OS X with Mail, 
 there is no such thing as a mail format change during the life of a major OS 
 version. So major 
 OS versions are 10.7, 10.8, and now 10.9

and you have a warranty for that?

 So it's impossible the mail format would change between 10.7.1 and 10.7.2 in 
 such a way that 10.7.2 
 mail could not be read by the 10.7.1 or 10.7.0 mail application

and you have a warranty for that?
no you do *not*

 I can upgrade and downgrade all along 10.7.x and the file format is the same.

you *think* that

 Recently I learned that there are two mail formats. There's the every day 
 used format that 
 is apparently completely incompatible between major versions of Mail
 I can export 10.8 Mail in this format, and 10.7 Mail can also read it. And 
 even this pissed 
 me off as well as several thousand users (at least) based on Apple's 
 community forums on the 
 subject because most of us expect to be able to directly import 10.7 mail 
 into 10.8 Mail. 

where you prove that what you said above is wrong

 Well, the mail servers regularly get updated by the company I pay for such 
 things, and I've 
 never noticed the change. It uses IMAP so I don't think the server even 
 cares, its just a bunch 
 of folders and files

blabla - nobody talks about the mailserver

the topic is *internal* data of *local* software
you may have luck and nothing happens

with bad luck you even won't realize that there are new mails you never face
because of happy upgrade/downgrade internal caches are accessed with
*undefined bahvior*

any software on that planet will recognize upgrades and convert *internal* data
but nobody will give you a warranty how the same software behaves after a 
downgrade

yes, in most cases nothing bad happens
in rare cases you recognize the problem and find a solution
in some cases you even don't recognize that internal things are slightly going 
wrong



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 11:41 AM, Simo Sorce s...@redhat.com wrote:

 I never said it won't work in absolute, it probably will work ok in many
 cases, just to cause incredible issues in others.
 
 It is a fine tool in the hands of an expert that knows how to check
 whether reverting to a snapshot is safe.

Why is the snapshot case any different from a user who reverts doing a clean 
install or yum downgrade?


 
 It is not going to be a good solution for non-expert users though
 *unless* you provide system APIs that *all* applications use to signal
 when they are doing irreversible changes so that the user can be warned
 about potential data loss right when he asks the system to revert a
 snapshot.

Developers should not be sneak attacking non-expert users with file format 
changes that aren't well announced in advance of consequences they probably 
won't be able to read their data if they downgrade the application.



Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 26.01.2014 21:13, schrieb Chris Murphy:
 On Jan 26, 2014, at 11:41 AM, Simo Sorce s...@redhat.com wrote:
 
 I never said it won't work in absolute, it probably will work ok in many
 cases, just to cause incredible issues in others.

 It is a fine tool in the hands of an expert that knows how to check
 whether reverting to a snapshot is safe.
 
 Why is the snapshot case any different from a user who reverts doing a clean 
 install or yum downgrade?

because the snapshot restores *a whole filesystem* and not only the affected 
application?

* restore a snapshot of /usr and you have fun with /var/lib/rpm
* restore a snapshot of /var/lib/ without /usr and you have fun with the rpmdb 
and others
* restore a snapshot of /usr without /etc and you *may have* random fun

and there are *hundrets* of such combinations where the last thing you
really would want is restore a snapshot because you have no plan about
the real-world impact in doing so

 It is not going to be a good solution for non-expert users though
 *unless* you provide system APIs that *all* applications use to signal
 when they are doing irreversible changes so that the user can be warned
 about potential data loss right when he asks the system to revert a
 snapshot.
 
 Developers should not be sneak attacking non-expert users with file format 
 changes that aren't well 
 announced in advance of consequences they probably won't be able to read 
 their data if they downgrade 
 the application

the perfect world won't happen, sad but true



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

On Sun, 2014-01-26 at 13:13 -0700, Chris Murphy wrote:
 On Jan 26, 2014, at 11:41 AM, Simo Sorce s...@redhat.com wrote:
 
  I never said it won't work in absolute, it probably will work ok in many
  cases, just to cause incredible issues in others.
  
  It is a fine tool in the hands of an expert that knows how to check
  whether reverting to a snapshot is safe.
 
 Why is the snapshot case any different from a user who reverts doing a
 clean install or yum downgrade?

It is not.

  It is not going to be a good solution for non-expert users though
  *unless* you provide system APIs that *all* applications use to signal
  when they are doing irreversible changes so that the user can be warned
  about potential data loss right when he asks the system to revert a
  snapshot.
 
 Developers should not be sneak attacking non-expert users with file
 format changes that aren't well announced in advance of consequences
 they probably won't be able to read their data if they downgrade the
 application.

Users should not expect miracles either.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 12:51 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 Am 26.01.2014 20:45, schrieb Chris Murphy:
 So ?
 It is only visible if you downgrade which a lot of software do not
 support and explicitly so
 
 The right way to do file format changes is you design the new format. 
 And in a minor version update, the application gains the ability to 
 read the new file format, but still writes the old file format. 
 The major version upgrade of the application is enabled to write the 
 new file format, while it can read either old or new formats.
 
 please look at the hidden folders in your userhome and /var/lib/
 to get a picture about what we are talking here

This sounds like FUD and there's no actual real world example. You're 
suggesting that if I have gnome-shell 3.10.3 and I either yum downgrade to 
3.10.1, or I do a clean install of Fedora 20 and get gnome-shell 3.10.0, that 
I'm going to see explosions? What's going to happen? Surely there aren't such 
significant configuration format changes between such minor versions, and 
surely the development teams anticipate this very use case where uses have a 
/home with such files, and have no choice but to revert to an older system with 
the same major version but lower minor version.

This is why changing configuration formats is hopefully a conscientious 
decision and not done willy nilly. From many years of experience I know I can 
reliably upgrade and downgrade at will, within minor versions of OS X - that is 
all versions of 10.7.x configuration file wise are expected to be compatible. 
And I'd exclude disposable cache files which by default aren't even backed up 
anyway as they're expected to be rebuilt on a restore.

Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 26.01.2014 21:30, schrieb Chris Murphy:
 On Jan 26, 2014, at 12:51 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Am 26.01.2014 20:45, schrieb Chris Murphy:
 So ?
 It is only visible if you downgrade which a lot of software do not
 support and explicitly so

 The right way to do file format changes is you design the new format. 
 And in a minor version update, the application gains the ability to 
 read the new file format, but still writes the old file format. 
 The major version upgrade of the application is enabled to write the 
 new file format, while it can read either old or new formats.

 please look at the hidden folders in your userhome and /var/lib/
 to get a picture about what we are talking here
 
 This sounds like FUD and there's no actual real world example

* i do not know what *may happen* by restore a snapshot
* you do not know what *may* happen by restore a snapshot
* nobody knows

why?

because nobody *can* know what exactly packages, versions are installed
in which combination or which *user specific* data may exist on exactly
the FS which is restored *additionally* to what the system sofware knows

frankly you can have your kwallet or the files your browser stores
passwords you recently created and thought they are safe on exactly
that FS, and they *maybe* saved *between* upgrade, realize a problem
and restore the snapshot

again: *nobody* knows for sure the *complete possible impact* on the
users computer by restore a snapshot because a upgrade should be
rolled back

surely, you can do that, i and many other people won't do this now
nor in the future for good reasons and not knowing *exactly* any
possible impact of a operation is a *damned good* reason

nothing more to say about that topic because

* i *never* won't do that
* i *never* would use LVM
* i *never* use BTRFS
* so my environment even does not support that snapshots

why i won#t use BTRFS/LVM?

because my drives are a Linux RAID10 and i never re-installed my system from
scratch nor would i do that in the future and *because* not everybody is using
a storage even supports snapshots it would be a bad idea to rely on that



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

On Sun, 2014-01-26 at 12:45 -0700, Chris Murphy wrote:
 I still really have no idea what sorts of changes you're talking
 about.

I think you made it abundantly clear :-)

I am also sure what I wanted to convey to people that understand what I
am talking about is also clear. So I think the matter has been expressed
well enough for devel and I do not think we need to further pollute the
list.

Thank you,
Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 1:07 PM, Reindl Harald h.rei...@thelounge.net wrote:

 Am 26.01.2014 20:56, schrieb Chris Murphy:
 What about mail application change the format of the mail folders ?
 
 Good question because I experienced this recently. So the way Apple does 
 this on OS X with Mail, 
 there is no such thing as a mail format change during the life of a major OS 
 version. So major 
 OS versions are 10.7, 10.8, and now 10.9
 
 and you have a warranty for that?

It's a long standing expectation and tradition, and considering millions of 
users had this sort of behavior occurred by now no doubt I'd have heard about 
it.

 
 I can upgrade and downgrade all along 10.7.x and the file format is the same.
 
 you *think* that

If there is a change, it's not a downward incompatible change.

 
 Recently I learned that there are two mail formats. There's the every day 
 used format that 
 is apparently completely incompatible between major versions of Mail
 I can export 10.8 Mail in this format, and 10.7 Mail can also read it. And 
 even this pissed 
 me off as well as several thousand users (at least) based on Apple's 
 community forums on the 
 subject because most of us expect to be able to directly import 10.7 mail 
 into 10.8 Mail. 
 
 where you prove that what you said above is wrong

No you just have reading comprehension problem. The minor versions are 
compatible. The major versions aren't.


 Well, the mail servers regularly get updated by the company I pay for such 
 things, and I've 
 never noticed the change. It uses IMAP so I don't think the server even 
 cares, its just a bunch 
 of folders and files
 
 blabla - nobody talks about the mailserver

Jerk. Simo said, in the line right above this that you cut: There are many 
other examples like this especially on the server side.



 the topic is *internal* data of *local* software
 you may have luck and nothing happens

This was not at all made clear from the start, it was assumed by people who 
understood. I explicitly asked if I was on the same page or not. Instead of 
bringing me up to speed, you decide to be condescending. Congratulations on 
your rudeness.


 with bad luck you even won't realize that there are new mails you never face
 because of happy upgrade/downgrade internal caches are accessed with
 *undefined bahvior*

Email are user documents the same as a Libreoffice document. You do not get to 
say that just because it's a semi-hidden database, that its file format is 
allowed to change in a downward incompatible manner. I will disagree with that 
position at every possible turn as something between sloppy programming and 
dereliction.

 
 any software on that planet will recognize upgrades and convert *internal* 
 data
 but nobody will give you a warranty how the same software behaves after a 
 downgrade

Well insofar as the whole software EULA paradigm basically says for any reason, 
willful or not, they can blow up your data in any direction possible and there 
is no liability claim whatsoever… what you're saying doesn't even apply to 
upgrades.

 yes, in most cases nothing bad happens
 in rare cases you recognize the problem and find a solution
 in some cases you even don't recognize that internal things are slightly 
 going wrong

It's no worse a risk than a conventional reversion with a clean install. So I 
fail to see how any of this relates to snapshots.


Chris Murphy
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 26.01.2014 21:56, schrieb Chris Murphy:
 On Jan 26, 2014, at 1:07 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Well, the mail servers regularly get updated by the company I pay for such 
 things, and I've 
 never noticed the change. It uses IMAP so I don't think the server even 
 cares, its just a bunch 
 of folders and files

 blabla - nobody talks about the mailserver
 
 Jerk. Simo said, in the line right above this that you cut: There are many 
 other examples like this especially on the server side.

be careful in which context you somebody calls a Jerk

 the topic is *internal* data of *local* software
 you may have luck and nothing happens
 
 This was not at all made clear from the start, it was assumed by people who 
 understood

because that people thought somebody with that much replies
to the thread would have understodd the topic

 I explicitly asked if I was on the same page or not. Instead of bringing me 
 up to speed, 
 you decide to be condescending. Congratulations on your rudeness

as you can see some lines above you needed *exactly* that way of comminucation
to understand what we are talking about in this thread - this is the *dvel* list
and so technical understanding is implicit in a discussion

 with bad luck you even won't realize that there are new mails you never face
 because of happy upgrade/downgrade internal caches are accessed with
 *undefined bahvior*
 
 Email are user documents the same as a Libreoffice document. You do not get 
 to say that just 
 because it's a semi-hidden database, that its file format is allowed to 
 change in a downward 
 incompatible manner

what exactly did you not understand in the two words internal caches
frankly i faced mail clients where you needed to remove the complete
IMAP account to stop not display any new or moved message in specific
folders

 any software on that planet will recognize upgrades and convert *internal* 
 data
 but nobody will give you a warranty how the same software behaves after a 
 downgrade
 
 Well insofar as the whole software EULA paradigm basically says for any 
 reason, willful 
 or not, they can blow up your data in any direction possible and there is no 
 liability 
 claim whatsoever… what you're saying doesn't even apply to upgrades.

google for undefined behavior

 yes, in most cases nothing bad happens
 in rare cases you recognize the problem and find a solution
 in some cases you even don't recognize that internal things are slightly 
 going wrong
 
 It's no worse a risk than a conventional reversion with a clean install

well, i never re-installed any linux system in my life for good reasons
the same reasons i never would restore a snapshot of my whole filesystem
or even more worse *a complete tree* alone of it

 So I fail to see how any of this relates to snapshots

that you fail to see the possible impact of a snapshot-restore is obviously
and you do not need to repeat that again and again




signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 26.01.2014 21:56, schrieb Chris Murphy:
 No you just have reading comprehension problem. The minor versions are 
 compatible. The major versions aren't

one last question: what are firefox updates 25-26-27
minor, major, dunno?

more and more software has no minor/major splitting at all
systemd, firefox, chrome..

what warranties do you have?
none!

what warranties did you ever have?
none as long the specific devloper did not make any promises
luck is no warranty as well as until now is not



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 1:18 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 
 Am 26.01.2014 21:13, schrieb Chris Murphy:
 On Jan 26, 2014, at 11:41 AM, Simo Sorce s...@redhat.com wrote:
 
 I never said it won't work in absolute, it probably will work ok in many
 cases, just to cause incredible issues in others.
 
 It is a fine tool in the hands of an expert that knows how to check
 whether reverting to a snapshot is safe.
 
 Why is the snapshot case any different from a user who reverts doing a clean 
 install or yum downgrade?
 
 because the snapshot restores *a whole filesystem* and not only the affected 
 application?

If I knew the problem was with a particular affected application, why would I 
be using a snapshot rollback approach or clean install rather than a yum 
downgrade app approach?
 
 * restore a snapshot of /usr and you have fun with /var/lib/rpm
 * restore a snapshot of /var/lib/ without /usr and you have fun with the 
 rpmdb and others
 * restore a snapshot of /usr without /etc and you *may have* random fun
 
 and there are *hundrets* of such combinations where the last thing you
 really would want is restore a snapshot because you have no plan about
 the real-world impact in doing so

Well what sort of moron would do rollbacks like this? You're saying if someone 
puts a stick of dynamite in their mouth then ZOMG! going to die!, but not 
accounting for why they would put dynamite in their mouth in the first place. 
This is simply not how rollbacks are done. Yes there are hundreds of 
mindnumbingly stupid ways a user could break their system. No one is 
recommending rollbacks that work the way you describe.


Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 27.01.2014 00:26, schrieb Chris Murphy:
 On Jan 26, 2014, at 1:18 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Am 26.01.2014 21:13, schrieb Chris Murphy:
 On Jan 26, 2014, at 11:41 AM, Simo Sorce s...@redhat.com wrote:

 I never said it won't work in absolute, it probably will work ok in many
 cases, just to cause incredible issues in others.

 It is a fine tool in the hands of an expert that knows how to check
 whether reverting to a snapshot is safe.

 Why is the snapshot case any different from a user who reverts doing a 
 clean install or yum downgrade?

 because the snapshot restores *a whole filesystem* and not only the affected 
 application?
 
 If I knew the problem was with a particular affected application, why would I 
 be using a snapshot rollback approach or clean install rather than a yum 
 downgrade app approach?

 * restore a snapshot of /usr and you have fun with /var/lib/rpm
 * restore a snapshot of /var/lib/ without /usr and you have fun with the 
 rpmdb and others
 * restore a snapshot of /usr without /etc and you *may have* random fun

 and there are *hundrets* of such combinations where the last thing you
 really would want is restore a snapshot because you have no plan about
 the real-world impact in doing so
 
 Well what sort of moron would do rollbacks like this? You're saying if 
 someone puts a stick of dynamite in their mouth then ZOMG! going to die!, but 
 not accounting for why they would put dynamite in their mouth in the first 
 place. This is simply not how rollbacks are done. Yes there are hundreds of 
 mindnumbingly stupid ways a user could break their system. No one is 
 recommending rollbacks that work the way you describe.

do yourself and everybody a favour and

* don't claim others are rude while you talk like above and worser half of the 
thread
* don't talk about things above your technical scope
* discuss with software engineers while lacking basic understanding of the topic

posts like yours in that thread belongs to the users list and *not* to a 
development list



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 1:40 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 
 Am 26.01.2014 21:30, schrieb Chris Murphy:
 On Jan 26, 2014, at 12:51 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Am 26.01.2014 20:45, schrieb Chris Murphy:
 So ?
 It is only visible if you downgrade which a lot of software do not
 support and explicitly so
 
 The right way to do file format changes is you design the new format. 
 And in a minor version update, the application gains the ability to 
 read the new file format, but still writes the old file format. 
 The major version upgrade of the application is enabled to write the 
 new file format, while it can read either old or new formats.
 
 please look at the hidden folders in your userhome and /var/lib/
 to get a picture about what we are talking here
 
 This sounds like FUD and there's no actual real world example
 
 * i do not know what *may happen* by restore a snapshot
 * you do not know what *may* happen by restore a snapshot
 * nobody knows

Great, well I'll tell you what. I will just keep living dangerously, and when I 
find a real world case of this, I'll file a bug. How about that?

 because nobody *can* know what exactly packages, versions are installed
 in which combination or which *user specific* data may exist on exactly
 the FS which is restored *additionally* to what the system sofware knows
 
 frankly you can have your kwallet or the files your browser stores
 passwords you recently created and thought they are safe on exactly
 that FS, and they *maybe* saved *between* upgrade, realize a problem
 and restore the snapshot

Oh for f's sake. And *maybe* the next time I take a shower I'll slip and fall, 
so I'll just decide now to not bathe ever again. This hysterical paranoia 
you're going on about is even less hypothetical than slipping in the bathroom. 
I read this buttkiss nonsense and feel like someone has injected my brain with 
novocaine. 


 again: *nobody* knows for sure the *complete possible impact* on the
 users computer by restore a snapshot because a upgrade should be
 rolled back.

Well you know what I think, is that applications should largely be self 
contained instead of sneezing all kinds of crap all over my file system. It's 
one of the best examples of why I prefer using OS X compared to Windows, which 
are drag and drop installation of applications that don't install weird junk 
all over my computer. Very very easy to rollback from this.



 
 surely, you can do that, i and many other people won't do this now
 nor in the future for good reasons and not knowing *exactly* any
 possible impact of a operation is a *damned good* reason
 
 nothing more to say about that topic because
 
 * i *never* won't do that
 * i *never* would use LVM
 * i *never* use BTRFS
 * so my environment even does not support that snapshots

Uh huh. So this is sort like a user coming onto this forum and talking trash 
about all of linux and Fedora and what all is broken and doesn't fit their use 
case or workflow at all, and then after 50 f'n emails they say they never use 
linux or Fedora. Even for you this is an especially egregious waste of time and 
brain cells. I can even feel the rot occurring in my brain from reading this 
mindnumbing   nonsense you've written in this whole thread, and the icing on 
the cake is that you don't even use the technologies you're bitching about. 
Bitch, bitch, bitch. That's the only thing you've accomplished. You're just 
bitching. It's f'n annoying.

 
 why i won#t use BTRFS/LVM?
 
 because my drives are a Linux RAID10 and i never re-installed my system from
 scratch nor would i do that in the future and *because* not everybody is using
 a storage even supports snapshots it would be a bad idea to rely on that

I think you having access to a computer with internet access is a bad idea.


Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


Am 27.01.2014 00:41, schrieb Chris Murphy:
 Great, well I'll tell you what. I will just keep living dangerously, and when 
 I find a real world case of this, I'll file a bug. How about that?

do that, your problem

 because nobody *can* know what exactly packages, versions are installed
 in which combination or which *user specific* data may exist on exactly
 the FS which is restored *additionally* to what the system sofware knows

 frankly you can have your kwallet or the files your browser stores
 passwords you recently created and thought they are safe on exactly
 that FS, and they *maybe* saved *between* upgrade, realize a problem
 and restore the snapshot
 
 Oh for f's sake. And *maybe* the next time I take a shower I'll slip and fall

off-topic

you do not understand anything this theard is about so why not leaves us in 
peace?

 It's one of the best examples of why I prefer using OS X compared to Windows

then use it

 nothing more to say about that topic because

 * i *never* won't do that
 * i *never* would use LVM
 * i *never* use BTRFS
 * so my environment even does not support that snapshots
 
 Uh huh. So this is sort like a user coming onto this forum and talking trash 
 about all of linux and Fedora and what all is broken and doesn't fit their 
 use 
 case or workflow at all, and then after 50 f'n emails they say they never use 
 linux or Fedora

you do not understand the intention of that thread at all
so why you don't just listen and be quite?

 I think you having access to a computer with internet access is a bad idea

must be why i get paid for server-adminstration and security for a decade...
please bother the users-lists where i am no longer subscribed because people
like you leading to get upset again and again




signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 4:29 PM, Reindl Harald h.rei...@thelounge.net wrote:

 do yourself and everybody a favour and
 
 * don't claim others are rude while you talk like above and worser half of 
 the thread
 * don't talk about things above your technical scope
 * discuss with software engineers while lacking basic understanding of the 
 topic
 
 posts like yours in that thread belongs to the users list and *not* to a 
 development list

Request declined.

You are the only person who has suggested a method of rollbacks that 
fundamentally would not work, and then argued against it. You created what's 
referred to as, a strawman argument. Also known as being full of crap. So you 
can take your silly logical fallacies and mock victimization and stick 'em 
where the sun don't shine, Reindl. Put it all right back where you got your 
ridiculous snapshot-rollback concept.

Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 27.01.2014 00:51, schrieb Chris Murphy:
 On Jan 26, 2014, at 4:29 PM, Reindl Harald h.rei...@thelounge.net wrote:
 
 do yourself and everybody a favour and

 * don't claim others are rude while you talk like above and worser half of 
 the thread
 * don't talk about things above your technical scope
 * discuss with software engineers while lacking basic understanding of the 
 topic

 posts like yours in that thread belongs to the users list and *not* to a 
 development list
 
 Request declined.
 
 You are the only person who has suggested a method of rollbacks that 
 fundamentally would not work

are you drunken?

i have *not* requested any method of rollback

i just gave a few warnings what problems has to be considered if rollbacks
of snapshots are taken as possible solution - so *stop* talk to threads if
you have no clue about the topic and about who said what

* read what others said
* start at the begin of the thread doing that
* try to understand what you read before commetn any word
* look at the *context* of several posts becaus eoyu need that information to 
understand
* after that claim what people suggested or in reality you would say nothing at 
all




signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

2014-01-26 Thread Kevin Fenzi

I don't think this subthread is being particularly useful... 

And the personal attacks are undesirable. 

Please stop or at least take it to private email. 

Thanks,

kevin


signature.asc
Description: PGP signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 4:47 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 Am 27.01.2014 00:41, schrieb Chris Murphy:
 Great, well I'll tell you what. I will just keep living dangerously, and 
 when I find a real world case of this, I'll file a bug. How about that?
 
 do that, your problem
 
 because nobody *can* know what exactly packages, versions are installed
 in which combination or which *user specific* data may exist on exactly
 the FS which is restored *additionally* to what the system sofware knows
 
 frankly you can have your kwallet or the files your browser stores
 passwords you recently created and thought they are safe on exactly
 that FS, and they *maybe* saved *between* upgrade, realize a problem
 and restore the snapshot
 
 Oh for f's sake. And *maybe* the next time I take a shower I'll slip and fall
 
 off-topic
 
 you do not understand anything this theard is about so why not leaves us in 
 peace?

I'll add rampant hyperbole to your list of personal attributes. I'm the one who 
doesn't understand anything even though you don't use or have any use for 
snapshots, and you also want no one to have them or it'll make developers 
careless:

On Jan 25, 2014, at 6:10 PM, Reindl Harald h.rei...@thelounge.net wrote:

 the short version of ahwat you said could have been forget snapshots at all 
 to solve
 such problems to not lead dvelopers into temptation of i can be less caeful 
 because
 we have snapshots
 
 in other words: don't work around problems by create new ones 


And then you propose a ridonkulous snapshot-rollback strategy that would for 
certain cause major problems if the rollback were actually done, and then use 
that as fait accompli for why the entire concept of fs rollbacks are stupid. 
Your arguments are asinine. Your emails belong in a kill file.


Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 27.01.2014 01:07, schrieb Chris Murphy:

 And then you propose a ridonkulous snapshot-rollback strategy that would for 
 certain cause major problems 
 if the rollback were actually done

*the opposite is true because i WARNED of doing snapshots*



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 27.01.2014 00:57, schrieb Kevin Fenzi:
 I don't think this subthread is being particularly useful... 
 
 And the personal attacks are undesirable. 
 Please stop or at least take it to private email

*sorry* for not early enough realize trolling in first start with
the same argumentation as Simon and me to later fight against it
while now claim i came up with the idea of snapshots while
warning all the time and tried to explain Chris *why* i
warn

 Original-Nachricht 
Betreff: Re: Drawing lessons from fatal SELinux bug #1054350
Datum: Sat, 25 Jan 2014 16:42:13 -0700
Von: Chris Murphy li...@colorremedies.com
Antwort an: Development discussions related to Fedora 
devel@lists.fedoraproject.org
An: Development discussions related to Fedora devel@lists.fedoraproject.org

I don't follow this. The realization an update is bad doesn't necessarily occur 
right away. So we still need a way
to separate system domain vs user domain, at least, so that system files are 
rolled back separately from user files

___

can someone *please stop that troll telling lies*

 And then you propose a ridonkulous snapshot-rollback strategy that would for 
 certain cause
 major problems if the rollback were actually done, and then use that as fait 
 accompli for
 why the entire concept of fs rollbacks are stupid. Your arguments are 
 asinine. Your emails
 belong in a kill file.



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 4:55 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 
 Am 27.01.2014 00:51, schrieb Chris Murphy:
 On Jan 26, 2014, at 4:29 PM, Reindl Harald h.rei...@thelounge.net wrote:
 
 do yourself and everybody a favour and
 
 * don't claim others are rude while you talk like above and worser half of 
 the thread
 * don't talk about things above your technical scope
 * discuss with software engineers while lacking basic understanding of the 
 topic
 
 posts like yours in that thread belongs to the users list and *not* to a 
 development list
 
 Request declined.
 
 You are the only person who has suggested a method of rollbacks that 
 fundamentally would not work
 
 are you drunken?

I haven't had anything to drink in 24+ hours, but this seems off topic.
 
 i have *not* requested any method of rollback

You gave several examples of rollback-snapshot methods - same thing as you 
suggested them. I never said you requested them.

 
 i just gave a few warnings what problems has to be considered if rollbacks
 of snapshots are taken as possible solution - so *stop* talk to threads if
 you have no clue about the topic and about who said what

Yes, problems as a result of improper rollback methods. I will not stop 
challenging junk suggestions which you then use to cast a wide net to argue 
against all forms of snapshotting and rollback. It's absurd argumentation.


 * read what others said
 * start at the begin of the thread doing that
 * try to understand what you read before commetn any word
 * look at the *context* of several posts becaus eoyu need that information to 
 understand
 * after that claim what people suggested or in reality you would say nothing 
 at all

Take your own advice. I've been following the thread from the very start, and 
your snapshot-rollback examples are all junk. Just for instance:

On Jan 26, 2014, at 1:18 PM, Reindl Harald h.rei...@thelounge.net wrote:

 * restore a snapshot of /usr and you have fun with /var/lib/rpm
 * restore a snapshot of /var/lib/ without /usr and you have fun with the 
 rpmdb and others
 * restore a snapshot of /usr without /etc and you *may have* random fun


Only you have come up with such utterly implausible examples of 
snapshot-rollback behavior and then chosen to argue against *all* possible 
snapshot-rollbacks in general. No one would do rollbacks the way you describe 
above. It would almost certainly break the system.


Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 27.01.2014 01:18, schrieb Chris Murphy:
 On Jan 26, 2014, at 4:55 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Am 27.01.2014 00:51, schrieb Chris Murphy:
 On Jan 26, 2014, at 4:29 PM, Reindl Harald h.rei...@thelounge.net wrote:

 do yourself and everybody a favour and

 * don't claim others are rude while you talk like above and worser half of 
 the thread
 * don't talk about things above your technical scope
 * discuss with software engineers while lacking basic understanding of the 
 topic

 posts like yours in that thread belongs to the users list and *not* to a 
 development list

 Request declined.

 You are the only person who has suggested a method of rollbacks that 
 fundamentally would not work

 are you drunken?
 
 I haven't had anything to drink in 24+ hours, but this seems off topic.

 i have *not* requested any method of rollback
 
 You gave several examples of rollback-snapshot methods - same thing as you 
 suggested them. I never said you requested them

oh my god - i gave several examples *what could be dangerous* in doing that
i *never* ever suggested them
please re-read the thread and then come back with an excuse



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 5:10 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 
 Am 27.01.2014 01:07, schrieb Chris Murphy:
 
 And then you propose a ridonkulous snapshot-rollback strategy that would for 
 certain cause major problems 
 if the rollback were actually done
 
 *the opposite is true because i WARNED of doing snapshots*

Yes, you argued against the general case of snapshot-rollbacks while using bad 
examples of rollback methods that would obviously cause problems.


Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 5:20 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 
 Am 27.01.2014 01:18, schrieb Chris Murphy:
 On Jan 26, 2014, at 4:55 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Am 27.01.2014 00:51, schrieb Chris Murphy:
 On Jan 26, 2014, at 4:29 PM, Reindl Harald h.rei...@thelounge.net wrote:
 
 do yourself and everybody a favour and
 
 * don't claim others are rude while you talk like above and worser half 
 of the thread
 * don't talk about things above your technical scope
 * discuss with software engineers while lacking basic understanding of 
 the topic
 
 posts like yours in that thread belongs to the users list and *not* to a 
 development list
 
 Request declined.
 
 You are the only person who has suggested a method of rollbacks that 
 fundamentally would not work
 
 are you drunken?
 
 I haven't had anything to drink in 24+ hours, but this seems off topic.
 
 i have *not* requested any method of rollback
 
 You gave several examples of rollback-snapshot methods - same thing as you 
 suggested them. I never said you requested them
 
 oh my god - i gave several examples *what could be dangerous* in doing that
 i *never* ever suggested them
 please re-read the thread and then come back with an excuse


suggested them can mean two things in English: you recommend them, or they 
are examples. I mean the 2nd case. I understand that you were not ever 
recommending your examples. You were suggesting them as examples why snapshots 
in general are bad.

The problem is that your examples are crap. They're bad examples because they 
would break the system, therefore no one would actually do snapshots-rollbacks 
per your examples, unless they wanted to blow up their system.


Chris Murphy
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 27.01.2014 01:32, schrieb Chris Murphy:
 On Jan 26, 2014, at 5:20 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Am 27.01.2014 01:18, schrieb Chris Murphy:
 You gave several examples of rollback-snapshot methods - same thing as you 
 suggested them. I never said you requested them

 oh my god - i gave several examples *what could be dangerous* in doing that
 i *never* ever suggested them
 please re-read the thread and then come back with an excuse
 
 suggested them can mean two things in English: you recommend them, or they 
 are examples. I mean the 2nd case. I understand that you were not ever 
 recommending your examples. You were suggesting them as examples why 
 snapshots in general are bad.
 
 The problem is that your examples are crap. They're bad examples because they 
 would break the system, therefore no one would actually do 
 snapshots-rollbacks per your examples, unless they wanted to blow up their 
 system.

boah the fact therefore no one would actually do snapshots-rollbacks per your 
examples needs to be proven
i only just warned about cases where a rollback would do harm and to *make 
sure* that really no one would
do it without take care

so where is now the point you started a flamewar against me instead
be quite or say ok, that would be bad and hopefully does not happen

this is a *dvelopent dicussion* and the goal of it is to *prevent*
mistakes ever happen *before* they are implemented or widely used



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 26, 2014, at 5:37 PM, Reindl Harald h.rei...@thelounge.net wrote:

 
 
 Am 27.01.2014 01:32, schrieb Chris Murphy:
 On Jan 26, 2014, at 5:20 PM, Reindl Harald h.rei...@thelounge.net wrote:
 Am 27.01.2014 01:18, schrieb Chris Murphy:
 You gave several examples of rollback-snapshot methods - same thing as you 
 suggested them. I never said you requested them
 
 oh my god - i gave several examples *what could be dangerous* in doing that
 i *never* ever suggested them
 please re-read the thread and then come back with an excuse
 
 suggested them can mean two things in English: you recommend them, or they 
 are examples. I mean the 2nd case. I understand that you were not ever 
 recommending your examples. You were suggesting them as examples why 
 snapshots in general are bad.
 
 The problem is that your examples are crap. They're bad examples because 
 they would break the system, therefore no one would actually do 
 snapshots-rollbacks per your examples, unless they wanted to blow up their 
 system.
 
 boah the fact therefore no one would actually do snapshots-rollbacks per 
 your examples needs to be proven

Really? That seems like saying no one would stick dynamite in their mouth 
unless they wanted to die needs to be proven. I think it will only take a 
handful of such instances to convince most rational people this isn't a good 
course of action.


 i only just warned about cases where a rollback would do harm and to *make 
 sure* that really no one would
 do it without take care

That was my *entire* point going back around 36 hours ago…

 Chris Murphy wrote:
 If there is a directory that contains update and non-update related file
 changes, that's a problem. If there's segmentation, then this can be done.
 
 Clearly /home needs to be separate (it's OK to take a snapshot but just
 don't use it by default in a rollback) or we lose changes in /home in a
 rollback from the time of the snapshot to the time of the decision to
 rollback.
 
 Another possible case it's /etc/ where the either a package or the user
 could make changes during the update.



 so where is now the point you started a flamewar against me instead
 be quite or say ok, that would be bad and hopefully does not happen

I did in fact state your examples were FUD. Where the flaming starts is when 
you said blabla - nobody talks about the mailserver when Simo *had* just 
mentioned server side changes which is what I was responding to. And blabla 
is just f'n rude from the outset, so yeah I'm going to be a bit of a dick when 
someone is a.) condescending, b.) says no one said X when someone did in fact 
say X; and c.) deletes the reply where someone said X; and d.) proceeds with a 
dozen emails about how I'm the one not paying attention when I asked for 
context clarification and you decided to jump down my throat and it went 
downhill quickly from there.

I do mostly just monitor this list, for several years. When people jump on you, 
are you quiet? No, you jump right back and you argue like hell. So don't tell 
me that I should be quiet, or how I should respond. From my perspective you 
were picking a fight, so I decide to play along and maybe mine was a little bit 
disproportionate of a response, but don't play victim just because you got 
burned.



 this is a *dvelopent dicussion* and the goal of it is to *prevent*
 mistakes ever happen *before* they are implemented or widely used

Which is exactly why I've involved myself in a thread on snapshotting because 
unlike you, I have been doing snapshots and rollbacks with LVM and Btrfs for 
quite a few years. I'm aware that there are some challenges that users will 
likely face and development needs to account for these things so they aren't 
easily getting into trouble or confused about where their data is.

Snapshots are a reality, simply sticking our head in the sand for a feature 
people have been asking for is simply not the way forward. I am not suggesting 
at all that your workflow should change to include snapshots, so I ask that you 
have the courtesy to not claim with bad examples that snapshots generally are a 
bad idea that will hose user's systems and make developers lazy and careless. 
This is an entirely voluntary project, you are not required to participate in 
some aspect of its technology you don't use and seem to not even care about.



Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

Am 27.01.2014 02:11, schrieb Chris Murphy:
 i only just warned about cases where a rollback would do harm and to *make 
 sure* that really no one would
 do it without take care

 That was my *entire* point going back around 36 hours ago

and that is why i do not understand your turn around 180 degree
against Simo and me beause *we both supported* your initial
viewpoint until you started to claim all the cases are invalid

 I did in fact state your examples were FUD

with no reason to do so

 Where the flaming starts is when you said blabla - nobody talks about the 
 mailserver
 when Simo *had* just mentioned server side changes which is what I was 
 responding to

hmpf - read again - server side changes != mailservers
after that you told about Apple Mail and what not and then switeched to 
mailservers
my problem was that you truned 180 degree and fighted against any argument going
in the direction where restore of snapshots may be tricky and dangerous while 
you
orginally before the subject changed even said the same

 And blabla is just f'n rude from the outset

because i had already enough of the turn 180 degree around and your
again and again argumentation about user documents and that they
don't change their format while never said that with a single line

 so yeah I'm going to be a bit of a dick when someone is a.) condescending, 
 b.) says
 no one said X when someone did in fact say X

still: nobody said mailserver, but forget it

 and c.) deletes the reply where someone said X

server side changes doe snot imply change *the format* how mails itself are 
stored

 and d.) proceeds with a dozen emails about how I'm the one not paying 
 attention when I
 asked for context clarification and you decided to jump down my throat and it 
 went
 downhill quickly from there.

then you maybe should have asked *only* about clarification instead start
calling developers names if they would change the format of user documents
which was *never* part of the context

 I do mostly just monitor this list, for several years. When people jump on 
 you,
 are you quiet?

no, but i am not a dickhead and listen if people telling me that
talking about user documents is not any part of the discussion
in case of downgrades and internal environment data of applications
may have changed unnoticed

 No, you jump right back and you argue like hell. So don't tell me that I
 should be quiet, or how I should respond

and if you really would look you have noticed a difference

 From my perspective you were picking a fight

if that would be true i have called you a lot of names in the public
which i *really* avoided while with some replies you begged for it

 so I decide to play along

why?

 and maybe mine was a little bit disproportionate of a response

a little? come on!

 but don't play victim just because you got burned

please calm more down and re-read the whole thread including the point
Simo even gave up completly to try explain you the context

 Which is exactly why I've involved myself in a thread on snapshotting because 
 unlike you, I have been doing snapshots and rollbacks with LVM and Btrfs for 
 quite a few years. 

i statet that i do not use snapshots nor the graphical stuf fnor gnome to make
clear *i am not affected* of any decision in that direction but *care* about
others, otherwise the whole sub-thread would not have been relevant for me

 I'm aware that there are some challenges that users will likely face and 
 development
 needs to account for these things so they aren't easily getting into trouble 
 or confused 
 about where their data is.

which was my whole point

 Snapshots are a reality, simply sticking our head in the sand for a feature 
 people 
 have been asking for is simply not the way forward. I am not suggesting at 
 all that 
 your workflow should change to include snapshots, so I ask that you have the 
 courtesy 
 to not claim with bad examples that snapshots generally are a bad idea that 
 will hose 
 user's systems and make developers lazy and careless

i did not say anything about snapshots in general
the topic is snapshots in case of updates and make it easy to roll them back
this needs *a lot of special care* that is my whole point

 This is an entirely voluntary project, you are not required to participate in 
 some 
 aspect of its technology you don't use and seem to not even care about

sorry that i case about the project in general



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Richard W.M. Jones

On Fri, Jan 24, 2014 at 11:14:50AM -0800, Adam Williamson wrote:
 On Fri, 2014-01-24 at 19:26 +0100, Michael Schwendt wrote:
 
   * That update made it out to the stable updates! In other words, the 
   draconian Update Policies that were enacted in a vain attempt to prevent 
   such issues from happening utterly failed at catching this bug.
  
  Those policies are not draconian enough [1]. On erroneous belief that
  a +1 from three different testers would mean that the update has seen
  enough testing, the test update has been published with the default karma
  threshold of +3. The testers have failed. It's too simple for testers to
  rush through the voting in bodhi without testing the updates
  painstakingly. The faster the better has lead to a fatal mistake in
  this case.
 
 I think that's being unnecessarily harsh on the testers. It's not at all
 obvious to anyone that you ought to test update/install of another
 package in order to validate an update to selinux-policy-targeted .
 Hell, I don't do that.

Doesn't / can't AutoQA (or whatever we're calling it these days) pick
up the new package, install it in a VM, and run through some automated
tests:

 - Does Fedora still boot with this package added?
 - Does GNOME still come up?
 - Does yum still work?

At least the third one might have automatically found this bug.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Bruno Wolff III


On Fri, Jan 24, 2014 at 20:40:28 -0800,
  Josh Stone jist...@redhat.com wrote:


My point was not about what root can do.  Suppose there's a vulnerable
'sudo' binary that gives everyone a root shell.  If that binary is
available on any executable path, even readonly, that's trouble.


That isn't true. File systems can be mounted such that suid bits are 
ignored. suid executables on such file systems are effectively just 
normal executables.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Ralf, Harald, you both actually mean the same thing, you're just 
misunderstanding each other due to inexact wording!

Yes, distro-sync will not remove packages which are not in the default-
enabled repositories at all (in any version) (nor will it downgrade them, 
obviously, because there is no version to downgrade them to).

And yes, distro-sync WILL downgrade packages if the new version is not (or 
not yet) available in a default-enabled repository.

It is clear that both of you know this, there was just a misunderstanding.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Adam Williamson

On Sat, 2014-01-25 at 10:43 +, Richard W.M. Jones wrote:

  I think that's being unnecessarily harsh on the testers. It's not at all
  obvious to anyone that you ought to test update/install of another
  package in order to validate an update to selinux-policy-targeted .
  Hell, I don't do that.
 
 Doesn't

No, it doesn't.

  / can't AutoQA (or whatever we're calling it these days) pick
 up the new package, install it in a VM, and run through some automated
 tests:
 
  - Does Fedora still boot with this package added?
  - Does GNOME still come up?
  - Does yum still work?

I answered precisely that question a couple of days ago in the other
thread:

https://lists.fedoraproject.org/pipermail/devel/2014-January/194155.html
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Chris Murphy wrote:
 If there is a directory that contains update and non-update related file
 changes, that's a problem. If there's segmentation, then this can be done.
 
 Clearly /home needs to be separate (it's OK to take a snapshot but just
 don't use it by default in a rollback) or we lose changes in /home in a
 rollback from the time of the snapshot to the time of the decision to
 rollback.
 
 Another possible case it's /etc/ where the either a package or the user
 could make changes during the update.

There's also /root, and then the most annoying case: /var. /var/lib/rpm 
definitely needs to be rolled back, but you DON'T want to roll back things 
such as log files in /var/log or systemwide databases (other than the RPM 
database).

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Tomasz Torcz

On Fri, Jan 24, 2014 at 03:10:04PM -0700, Chris Murphy wrote:
 
 On Jan 24, 2014, at 11:19 AM, Kevin Fenzi ke...@scrye.com wrote:
 
  On Fri, 24 Jan 2014 09:41:13 -0800
  Adam Williamson awill...@redhat.com wrote:
  
  AIUI there is/was a long-term plan to integrate this as core
  functionality using btrfs snapshots - in fact that was one of the
  major attractions of the idea of switching to btrfs-by-default in the
  first place. I believe those involved didn't think the LVM-based
  implementation was clean/robust enough to use by default, but a
  btrfs-based implementation would be. Do correct me if I'm wrong.
  
  I don't think snapshots are a partcularly good solution, unless there's
  some way to only roll back the rpm/yum transaction without also rolling
  back unrelated changes. 
 
 
 If there is a directory that contains update and non-update related file
 changes, that's a problem. If there's segmentation, then this can be done.  
 
 Clearly /home needs to be separate (it's OK to take a snapshot but just don't
 use it by default in a rollback) or we lose changes in /home in a rollback 
 from
 the time of the snapshot to the time of the decision to rollback.  
 
 Another possible case it's /etc/ where the either a package or the user could
 make changes during the update. Btrfs allows per file snapshots with cp
 --reflink so there might be a way to carve the snapshot with a scalpel but I
 prefer doing it with subvolume granularity. Plus that granularity translates 
 to
 LVM.  

  Note that this situation is perfectly handled by Offline Updates.
After reboot, there aren't collateral changes to filesystem, only 
upgrade-related
ones. So if there's a need for revert, the previous state is clearly defined.

-- 
Tomasz TorczThere exists no separation between gods and men:
xmpp: zdzich...@chrome.pl   one blends softly casual into the other.



pgpoWo_BV9He9.pgp
Description: PGP signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350



Am 25.01.2014 17:46, schrieb Tomasz Torcz:
 Note that this situation is perfectly handled by Offline Updates.
 After reboot, there aren't collateral changes to filesystem, only 
 upgrade-related
 ones. So if there's a need for revert, the previous state is clearly defined

says who?

UsrMove was as example forced with the excuse to support this as well
as /usr on a own partition beause one snapshot of the system


so and now imagine a common setup

* /boot
* /
* /var

have fun with restore your snapshot or / or /usr where you bomb the rootfs back
and the rpmdb is still like the restore never did happen




signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Sergio Pascual wrote:
 The situation (a broken system that cannot be upgraded)  could be
 mitigated a little bit by using yum + system snapshots. You can rollback
 to a previous sane system.

The big problem with that approach (other than the granularity issue already 
pointed out) is disk space. Even with a smart snapshotting technology that 
really only keeps on disk exactly what changed, it still requires a lot of 
extra disk space.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Adam Williamson wrote:

 On Fri, 2014-01-24 at 13:36 +0100, Kevin Kofler wrote:
 * If the package is already so screwed that it breaks the whole system,
 it cannot realistically get any worse.
 
 Sure it can. It can wipe all your data, or mail it to the NSA...

That's why I said realistically. What is the probability of something like 
that happening IN PRACTICE from a trivial change, especially one that 
reverts something to a known previous state? Perfect QA does not exist 
anyway, so it is all a matter of probabilities.

 Breaks the whole system is high on the Pantscon Scale, sure, but it's
 not the highest. Data loss and security compromise both come higher.

And you seriously think a week (or 2) of testing can catch that? Security 
vulnerabilities can take years to get noticed, creeping data loss days to 
weeks. The only thing in your catastrophe scale that can be noticed in 1-2 
weeks is blatant wipes entire directories immediately data loss, something 
extremely unlikely to happen from a regression fix (but so are the other 
catastrophic scenarios, unless the problem was already there before the 
regression fix and is thus no reason to withhold the fix).

 * A regression fix is usually a trivial change, often reverting something
 to a previous, already well-tested, state.
 
 Sure. And what could possibly go wrong.

The risk of a catastrophe as you described happening needs to be estimated 
(a tiny fraction of a percent) and weighed against the breakage (and denial 
of service, if security terms are the only ones you understand!) of keeping 
the broken update in stable for longer (and thus also letting it affect more 
users). I think it is blatantly obvious that the first theoretical risk is 
the much better one to take compared to the second, very practical one.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Adam Williamson wrote:
 The 'comment' field exists to allow people to express all these things,
 but as it's just a completely free-form text field, it's intrinsically
 impossible to really base any programmatic stuff or even policy on it.
 In theory maintainers could submit updates without using autokarma and
 then keep a careful eye on the feedback and 'tend' their updates
 manually, but I think it's pretty clear that in practice, this is not
 what happens: maintainers really want to be able to use the karma system
 as a 'helper', they want to farm out the evaluation process to Bodhi/the
 karma system. But our current system is too stupid to handle this
 perfectly, so we get these breakdowns.

It's not that they WANT to be able to do that, it's that the system is 
rigged to encourage that broken practice. Autokarma (ab)use was much lower 
before the enactment of the Update Policies. And really, autokarma needs to 
just go away entirely. Having an intelligent being interpret the free-form 
text field is the only way to make sane decisions (which also implies that 
we should not arbitrarily impose any restrictions that, by their nature, 
cannot take the free-form text into account).

 With a more flexible karma system we have a *lot* of opportunity to do
 much cleverer stuff. We can provide presets for all the above different
 things that are currently commonly expressed via +1 or -1 with a
 comment. This opens up possibilities at two different levels: the distro
 policy level, and the packager level. We can make the distro policy much
 more fine-grained, if we want to - we can require certain of the 'karma
 types' to be available in all updates, and for instance, block any
 update where X people pull the 'it's completely busted' or 'it
 introduces a security vulnerability' cord, regardless of how much
 broadly-categorized 'positive' karma it has. At the packager level, the
 packager gets the freedom to define a much more fine-grained policy for
 when they're happy that updates to their package are 'good to go', but
 they still don't have to sit there reading the emails and manually
 interpreting what people have written. You get to define the policy that
 makes the most sense for your package, within the confines of the
 distro-wide policy - if you have a good package-specific test suite, you
 can say to the auto-karma system 'don't send this update out until at
 least one person sets the I ran the test suite and it passed karma
 property.

To me, this just screams OVERENGINEERED!!!. :-(

You are introducing a lot of complexity, that will ultimately always only be 
an approximation to reality. You just cannot reliably quantify all the 
details. E.g. this introduces a regression, but the regression is that you 
sometimes have to click OK twice (instead of once) to format a 5.25 floppy 
in KFloppy, whereas it fixes a critical bug in KDE Plasma Desktop where all 
your data was sent to the NSA and then securely wiped from your hard disk. 
:-) (Yes, of course that is an exaggerated example. ;-) I sure hope we don't 
have bugs like that. ;-) ) If you only have fixes a bug, but introduces a 
regression as a feedback type, you probably end up making the wrong 
decision. If you try to get more fine-grained, then you again need numbers 
to quantify the severity of one issue vs. the other, and those will 
inherently be subjective. (Users always think THEIR bug is the end of the 
world whereas regressions that don't affect them are entirely unimportant.)

The complexity also means there are a lot more arbitrary parameters to deal 
with. The current stable/unstable thresholds are already bad enough, and 
often end up set to the wrong value. A decision process tends to be the 
worse the more arbitrary parameters it needs.

And of course, more complexity means less transparency. It becomes harder 
and harder to understand what really needs to be satisfied for an update to 
be allowed to go stable.

So I can only advocate for the KISS approach: The update is stable when the 
maintainer says so, period. We do not need any karma, be it a simple ±1 or a 
long (and inherently non-exhaustive) list of all the things that can 
possibly happen. So let's just do away with the 3 radio buttons and use a 
free-form text field only. We just need somebody able to read comments, and 
that is what we have maintainers for!

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Dominick Grift wrote:
 Sure, what i am saying is that this could have been prevented if the
 team just put a little more passion into it and also did some proof
 reading/coordination. The team knows whats going on. They know the
 issues and they can quickly and effortlessly identify issues like these
 if only they would take some time to watch each others commits.

Looking at the history of the involved bugs, using manual pushes rather than 
the broken karma automatism and taking into account Bugzilla comments, not 
just Bodhi comments, would probably also have prevented this fiasco. One of 
the bugs (not the one that ended up becoming the canonical bug, but an 
earlier one) was reassigned to selinux-policy fairly quickly.

One of the major flaws in the Bodhi karma system is that it cannot possibly 
see what happens in Bugzilla.

 Never the less, I think this issue could have been prevented even before
 a package was spun.

Yes, by disabling SELinux by default. :-)

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Michael Schwendt wrote:
 By the time the first testers noticed the scriptlet errors it was too
 late, since stable updates cannot be withdrawn.

That is also not a law of Physics. In the early days of Bodhi, one could 
actually unpush stuff from stable. Having stable updates become immutable is 
purely a policy decision. Withdrawing faulty updates has been done in the 
past (even after Bodhi stopped allowing it in the normal case; the pulling 
has then been done by an admin) and should be done again. Of course it won't 
fix the systems that already got upgraded, but it will (within mirroring 
delays) stop MORE systems from getting affected (and those that did already 
get the faulty update won't notice the difference, unless they distro-sync, 
in which case withdrawing the update actually fixes them, so in no case does 
it make things worse for them).

And I don't see any valid reason why stable updates cannot simply be 
withdrawn or sent back to testing by the maintainer. The update notes should 
also remain editable, so that bug references can be added when the bug was 
only found to be fixed after the stable push, errors in the update 
description can be fixed, etc.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Adam Williamson wrote:
 Yup, indeed. Of course, this is another area where we could improve the
 tooling: it doesn't seem like it'd be difficult for maintainers to be
 allowed to set a minimum timeframe before their update goes stable, but
 at present this isn't possible.

Why do we need to keep polishing that karma turd instead of just flushing it 
away? Especially karma automatism is totally broken by design.

 Again, hate to sound like a broken record, but it's just hard to get
 enthusiastic about trying to twiddle the edges of the process when the
 process is fundamentally inadequate.

Oh yes, it is! So let's do away with it!

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Michael Schwendt wrote:
 More lessons to learn:
 
 https://admin.fedoraproject.org/updates/FEDORA-2013-23627
   Karma:  17
   Stable karma:   16 (!)
 
 It has reached the karma threshold 16 after ~5 days.
 And those have not been all testers.

That can work for yum, but if I set the stable karma to 16 for kfloppy, the 
release will reach its EOL without it getting there (unless somebody targets 
testers specifically at kfloppy to win the bet ;-) … and buys them floppy 
drives, too ;-) ). Heck, even if Fedora releases didn't have an EOL, you or 
me probably wouldn't live to see it go stable.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Josh Stone

On 01/25/2014 06:03 AM, Bruno Wolff III wrote:
 On Fri, Jan 24, 2014 at 20:40:28 -0800,
Josh Stone jist...@redhat.com wrote:

 My point was not about what root can do.  Suppose there's a vulnerable
 'sudo' binary that gives everyone a root shell.  If that binary is
 available on any executable path, even readonly, that's trouble.
 
 That isn't true. File systems can be mounted such that suid bits are 
 ignored. suid executables on such file systems are effectively just 
 normal executables.

Ok, sure, you can mount -o nosuid,noexec,nodev ... but this isn't the
default for btrfs subvolume paths AFAIK.  It needs to be a conscious
decision in whatever snapshot design we choose.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Dominick Grift

On Sat, 2014-01-25 at 19:10 +0100, Kevin Kofler wrote:

  Never the less, I think this issue could have been prevented even before
  a package was spun.
 
 Yes, by disabling SELinux by default. :-)
 

No, that is a different discussion. Disabling SELinux does nothing to
solve this. If anything, to me this is confirmation of why we need a
good SELinux implementation. If this would happen to any other component
then a good SELinux implementation could have contained the damage
caused by issues just like this one.

The SELinux experience can, in my view be improved, and i believe your
problem is not with SELinux itself but with how it is
configured/implemented by default.

I just believe that a little team coordination, and a little more care
can go a long way, and that that is likely more efficient than trying to
create tests that would catch all of the bugs which sounds like utopia
to me.

I am not saying that the tests can't be improved or that they should not
be improved. It's just that in this case a little bit more care and a
double check by another involved party would probably have prevent this,
and similar other issues, in my view.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Michael Schwendt

On Sat, 25 Jan 2014 19:29:12 +0100, Kevin Kofler wrote:

  https://admin.fedoraproject.org/updates/FEDORA-2013-23627
Karma:17
Stable karma: 16 (!)
  
  It has reached the karma threshold 16 after ~5 days.
  And those have not been all testers.
 
 That can work for yum, but if I set the stable karma to 16 for kfloppy,

Nobody suggested doing that for kfloppy or other packages, which hardly
ever get feedback in bodhi.

If a test update doesn't get any feedback in bodhi, what does that imply?

If you mark it stable after 7 days, you've tested it yourself for some
days, correct?

If the update doesn't refer to any bugzilla tickets, what does that mean?

 the 
 release will reach its EOL without it getting there (unless somebody targets 
 testers specifically at kfloppy to win the bet ;-) … and buys them floppy 
 drives, too ;-) ). Heck, even if Fedora releases didn't have an EOL, you or 
 me probably wouldn't live to see it go stable.

Almost funny, if it weren't possible to mark test updates as stable after
7 days.

It could be that nobody uses the package at all, so it would not a big
deal if an update (or upgrade?) took 7+ days to enter the updates repo. ;-p
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Michael Schwendt

On Sat, 25 Jan 2014 19:17:14 +0100, Kevin Kofler wrote:

  By the time the first testers noticed the scriptlet errors it was too
  late, since stable updates cannot be withdrawn.
 
 That is also not a law of Physics. In the early days of Bodhi, one could 
 actually unpush stuff from stable. 

Pointing that out doesn't make a difference. Obviously, I don't refer
to technical contraints. Even before bodhi, e.g., the Fedora Extras signers
could modify the master repo in an emergency situation.

 Having stable updates become immutable is purely a policy decision.

Sure.

 Withdrawing faulty updates has been done in the 
 past (even after Bodhi stopped allowing it in the normal case; the pulling 
 has then been done by an admin) and should be done again. Of course it won't 
 fix the systems that already got upgraded, but it will (within mirroring 
 delays) stop MORE systems from getting affected (and those that did already 
 get the faulty update won't notice the difference, unless they distro-sync, 
 in which case withdrawing the update actually fixes them, so in no case does 
 it make things worse for them).

Not sure that can be generalised. Distro-sync may downgrade packages.
We don't test downgrades of packages (scriptlets e.g.), and we don't test
downgrades of software either. We can't be sure downgraded software can
restore state at runtime after a previous upgrade may have touched
(= converted, renamed or replaced) config files or database files.
Downgrades could also affect dependencies and may make it necessary
to have a system update tool run distro-sync automatically. There are
enough users already, who play too much with --skip-broken instead of
reporting uninstallable updates/packages quickly.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

2014-01-25 Thread Simo Sorce

On Sat, 2014-01-25 at 17:46 +0100, Tomasz Torcz wrote:
 On Fri, Jan 24, 2014 at 03:10:04PM -0700, Chris Murphy wrote:
  
  On Jan 24, 2014, at 11:19 AM, Kevin Fenzi ke...@scrye.com wrote:
  
   On Fri, 24 Jan 2014 09:41:13 -0800
   Adam Williamson awill...@redhat.com wrote:
   
   AIUI there is/was a long-term plan to integrate this as core
   functionality using btrfs snapshots - in fact that was one of the
   major attractions of the idea of switching to btrfs-by-default in the
   first place. I believe those involved didn't think the LVM-based
   implementation was clean/robust enough to use by default, but a
   btrfs-based implementation would be. Do correct me if I'm wrong.
   
   I don't think snapshots are a partcularly good solution, unless there's
   some way to only roll back the rpm/yum transaction without also rolling
   back unrelated changes. 
  
  
  If there is a directory that contains update and non-update related file
  changes, that's a problem. If there's segmentation, then this can be done.  
  
  Clearly /home needs to be separate (it's OK to take a snapshot but just 
  don't
  use it by default in a rollback) or we lose changes in /home in a rollback 
  from
  the time of the snapshot to the time of the decision to rollback.  
  
  Another possible case it's /etc/ where the either a package or the user 
  could
  make changes during the update. Btrfs allows per file snapshots with cp
  --reflink so there might be a way to carve the snapshot with a scalpel but I
  prefer doing it with subvolume granularity. Plus that granularity 
  translates to
  LVM.  
 
   Note that this situation is perfectly handled by Offline Updates.
 After reboot, there aren't collateral changes to filesystem, only 
 upgrade-related
 ones. So if there's a need for revert, the previous state is clearly defined.

Sorry, but this is simply not true.

I would really like to DISABUSE people of the notion that automated (or
not) rollbacks can be easily done in bulk, by the magic wand of file
system snapshots.

The ONLY way to do that is if you do not care at all about user's data
and simply accept that a rollback will also remove user data.

The reason is simple: lot's of software *changes* data as part of its
normal functioning, including and often in rollback-incompatible ways.

You cannot assume that upgrading a program that uses a database X from
version A to B can still work if you keep database X unchanged and then
rollback from B to A. Lot of applications apply changes to database at
upgrade time, either in the rpm scriplets or automatically as soon as a
new version binary is run.

It is basically impossible to find applications that handle the case
where you downgrade, in any more graceful way than punting and failing
to start in the *good* case. In the bad case they start and trash the
database.

And by database, do not think SQL/NOSQL engines only, it can be any
simple dataset in a file, including configuration files in user's homes.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Simo Sorce

On Sat, 2014-01-25 at 14:32 -0500, Colin Walters wrote:
 On Sat, 2014-01-25 at 10:37 -0800, Josh Stone wrote:
 
  Ok, sure, you can mount -o nosuid,noexec,nodev ... but this isn't the
  default for btrfs subvolume paths AFAIK.  It needs to be a conscious
  decision in whatever snapshot design we choose.
 
 This is definitely an issue with the OSTree design, since everything
 shares a physical partition (you can choose whatever block storage you
 want) - it's just hard links.
 
 I just filed:
 https://bugzilla.gnome.org/show_bug.cgi?id=722984
 for this.  

I forgot by gnome bugzilla password (again) so before I forget:
do not use .files or such it quickly becomes a mess. If you need to
annotate this kind of things I humbly suggest you add an xattr to the
file namespaced to ostree.

Alternatively, if you do not want to touch the original file at all,
keep a separate database where you note all these things, it will make
for a faster lookup in case you need bulk operations instead of having
to troll the whole tree.

 But really, now that KDBus is on the way, we can start using it for
 system services to replace many setuid binaries, like unix_chkpwd
 without losing the auditing trail and such that old indirection via
 dbus-daemon required.  That's a subject for a different thread though.

This is a good point, but a number of binaries are that way for legacy
reasons, or come from upstreams that care for portability and can't rely
on dbus (yet), so I think you need to care for the problem anyway.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350


On Jan 24, 2014, at 9:40 PM, Josh Stone jist...@redhat.com wrote:

 On 01/24/2014 05:27 PM, Chris Murphy wrote:
 On Jan 24, 2014, at 4:16 PM, Josh Stone jist...@redhat.com wrote:
 This concerns me especially in the case of security updates -- for 
 example, a vulnerable setuid-root binary should be locked up tight!
 
 The organization question is valid. But sudo or root could just mount
 any subvolume. However, btrfs read-only snapshots can't be written to
 even by root. Naturally root could just create a rw snapshot of a ro
 snapshot and then delete the ro snapshot, but an audit probably ought
 to show the subvolume UUIDs and creation dates involved so that we'd
 know this is what happened.
 
 My point was not about what root can do.  Suppose there's a vulnerable
 'sudo' binary that gives everyone a root shell.  If that binary is
 available on any executable path, even readonly, that's trouble.


OK, so is the fact it's persistently available the problem? Because if I were 
to have a persistent backup of sysroot mounted, I've got the same attack vector 
available. By default for even an unprivileged user gnome-shell mounts with By 
default, gnome-shell mounts volumes with 
rw,nosuid,nodev,relatime,seclabel,uhelper=udisks2.

 
 As you say, LVM snapshots are out of view, but with btrfs it needs to be
 an inaccessible subvolume path, or mounted noexec, etc.

To make inaccessible: mount a subvol outside of the presently mounted path, 
snapshot, umount. 

Seems like I can independently mount subvolumes with noexec:

49 37 0:45 /isos /mnt/isos rw,relatime shared:35 - btrfs /dev/sdb 
rw,seclabel,compress=lzo,space_cache
177 37 0:45 /archive /mnt/root rw,noexec,relatime shared:159 - btrfs /dev/sdb 
rw,seclabel,compress=lzo,space_cache

So another possibility is to have a snapshots subvolume persistently mounted, 
with noexec, and always place snapshots in that subvolume.



Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Dominick Grift wrote:
 No, that is a different discussion.

Nonsense. That SELinux should be disabled is the whole point of this thread 
(I know, I have started it!), all the suggestions (in the various 
subthreads) of how to paper over the problem are off topic.

 Disabling SELinux does nothing to solve this.

Oh sure it does. It eliminates this whole class of breakage (critical 
components unable to do their job because SELinux gets in their way) once 
and for all. This type of breakage keeps occurring, in fact one instance is 
still ongoing in Rawhide while we're discussing this:
https://bugzilla.redhat.com/show_bug.cgi?id=1052317

Disable SELinux and nothing will keep those components (RPM, display 
managers, etc.) from doing their work.

 If anything, to me this is confirmation of why we need a good SELinux
 implementation. If this would happen to any other component then a good
 SELinux implementation could have contained the damage caused by issues
 just like this one.

You don't seem to understand at all what SELinux is (it is not a tool 
magically able to fix bugs, its only purpose is to keep programs from doing 
their work)…

 The SELinux experience can, in my view be improved, and i believe your
 problem is not with SELinux itself but with how it is
 configured/implemented by default.

… nor what it can or cannot do. (No amount of configuration can make SELinux 
do anything other than block (i.e. break) things.)

 I just believe that a little team coordination, and a little more care
 can go a long way, and that that is likely more efficient than trying to
 create tests that would catch all of the bugs which sounds like utopia
 to me.

What coordination?

 I am not saying that the tests can't be improved or that they should not
 be improved. It's just that in this case a little bit more care and a
 double check by another involved party would probably have prevent this,
 and similar other issues, in my view.

Sure, dropping autokarma could help (and should be done in any case, that 
Bodhi feature never made any sense), but ultimately there's no way around 
disabling SELinux. Enabling it by default in Fedora has always been a 
mistake!

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Michael Schwendt wrote:
 If the update doesn't refer to any bugzilla tickets, what does that mean?

In that particular case, it means that we are updating all the KDE software 
compilation and so there's a new release of KFloppy too, which most likely 
doesn't even contain any actual changes from upstream (just a new version 
number on the tarball), but the updates are scripted, and the version bump 
is also needed to keep our metapackages (kdeutils in this case) working. :-)

That said, in practice, we file those as grouped updates and so there's a 
chance that the update actually gets some karma. Surely not because of 
KFloppy though. ;-)

 Almost funny, if it weren't possible to mark test updates as stable after
 7 days.

Right, but you were proposing to wait until it reaches a karma of +16.

 It could be that nobody uses the package at all, so it would not a big
 deal if an update (or upgrade?) took 7+ days to enter the updates repo.
 ;-p

But then the right solution is to disable karma automatism entirely, not to 
set it to some ridiculously high value.

Those meaningless thresholds need to go away (and really, the whole concept 
of Bodhi karma and the policies that depend on it).

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Dominick Grift

On Sat, 2014-01-25 at 21:51 +0100, Kevin Kofler wrote:
 Dominick Grift wrote:
  No, that is a different discussion.
 
 Nonsense. That SELinux should be disabled is the whole point of this thread 
 (I know, I have started it!), all the suggestions (in the various 
 subthreads) of how to paper over the problem are off topic.
 

Sorry, I must have misinterpreted the subject: Drawing lessons from
fatal SELinux bug

  Disabling SELinux does nothing to solve this.
 
 Oh sure it does. It eliminates this whole class of breakage (critical 
 components unable to do their job because SELinux gets in their way) once 
 and for all. This type of breakage keeps occurring, in fact one instance is 
 still ongoing in Rawhide while we're discussing this:
 https://bugzilla.redhat.com/show_bug.cgi?id=1052317
 
 Disable SELinux and nothing will keep those components (RPM, display 
 managers, etc.) from doing their work.

  If anything, to me this is confirmation of why we need a good SELinux
  implementation. If this would happen to any other component then a good
  SELinux implementation could have contained the damage caused by issues
  just like this one.
 
 You don't seem to understand at all what SELinux is (it is not a tool 
 magically able to fix bugs, its only purpose is to keep programs from doing 
 their work)…

I did not mean to suggest that. I meant to suggest that SELinux would be
able to contain the damage, referring to fatal in: Drawing lessons
from fatal SELinux bug

  The SELinux experience can, in my view be improved, and i believe your
  problem is not with SELinux itself but with how it is
  configured/implemented by default.
 
 … nor what it can or cannot do. (No amount of configuration can make SELinux 
 do anything other than block (i.e. break) things.)
 

Actually it is the other way around. SELinux blocks everything by
default. Everything needs to be explicitly allowed by means of
configuration

  I just believe that a little team coordination, and a little more care
  can go a long way, and that that is likely more efficient than trying to
  create tests that would catch all of the bugs which sounds like utopia
  to me.
 
 What coordination?

For example coordinate who does what where and when.

 
  I am not saying that the tests can't be improved or that they should not
  be improved. It's just that in this case a little bit more care and a
  double check by another involved party would probably have prevent this,
  and similar other issues, in my view.
 
 Sure, dropping autokarma could help (and should be done in any case, that 
 Bodhi feature never made any sense), but ultimately there's no way around 
 disabling SELinux. Enabling it by default in Fedora has always been a 
 mistake!
 
 Kevin Kofler
 


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350


Am 25.01.2014 22:00, schrieb Kevin Kofler:
 But then the right solution is to disable karma automatism entirely, not to 
 set it to some ridiculously high value.
 
 Those meaningless thresholds need to go away (and really, the whole concept 
 of Bodhi karma and the policies that depend on it)

i am not entirely sure how that is meant

* disable the automatism to push to stable
* forget the whole karma system at all

in case of disable the automatism to push to stable i agree

in my opinion karma is a indication for the maintainer but not
the decision - the karma has to be handeled differently for the
same package and different updates and only the maintainer can
decide that *as person*

why?

because it depends on the change itself

speaking with my developer hat on: there are updates on software
inside our company where i do not hestitate a single seconds deploy
the new CMS version to some hundrets of customers without tell anybody
there was a update at all because *i know* there can be no bad impact

on the other hand there are updates and changes which needs to prepare
any singel webhost, rollout a small update to prepare the real one by
add database colums not used currently but need to be there in the time
window files are replaced and database scheme can be updated

the second case is for not have any single request going wrong

and there is another category where all the work above has to be done
and tested thousands of times but still need a keep your eyes open
after it is done because you can't test and verify every single action
a complex software may do with every possible input data





signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-25 Thread Michael Schwendt

On Sat, 25 Jan 2014 22:00:02 +0100, Kevin Kofler wrote:

 Right, but you were proposing to wait until it reaches a karma of +16.

Certainly not. That Yum update is only a good example where a high
karma threshold has been reached in less than a week, and even without
a vote from all available/active testers. Sure, Yum is widely-used, which
cannot be said about niche-market packages.

I'm proposing that Test Updates are offered for a minimum duration
to allow for the time it takes to push them into the repo *and*
be picked up by world-wide mirrors. That increases the chance that
available testers get a chance of evaluating an update and giving
feedback _before_ it gets marked stable because of reaching +3 very
early (based on koji downloads, for example).

That has worked rather well for the more interesting update with a karma
threshold of 5: https://admin.fedoraproject.org/updates/FEDORA-2013-22706

The -1 votes have not been too late.

On the contrary, if a tester is available and finds a bug in a new 
update as offered on a nearby mirror, but in bodhi the update has been
marked stable in less than a day already or has skipped updates-testing
even, the tester cannot help anymore. That's a case that would be
avoidable.

  It could be that nobody uses the package at all, so it would not a big
  deal if an update (or upgrade?) took 7+ days to enter the updates repo.
  ;-p
 
 But then the right solution is to disable karma automatism entirely, not to 
 set it to some ridiculously high value.

Yes, debatable. But we shouldn't argue about it. '5' or '10' for Yum
updates isn't ridiculously high while still giving testers the opportunity
to vote cleverly and trigger the automatic push.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

2014-01-25 Thread Tomasz Torcz

On Sat, Jan 25, 2014 at 02:55:32PM -0500, Simo Sorce wrote:
 On Sat, 2014-01-25 at 17:46 +0100, Tomasz Torcz wrote:
   
   If there is a directory that contains update and non-update related file
   changes, that's a problem. If there's segmentation, then this can be 
   done.  
  
Note that this situation is perfectly handled by Offline Updates.
  After reboot, there aren't collateral changes to filesystem, only 
  upgrade-related
  ones. So if there's a need for revert, the previous state is clearly 
  defined.
 
 Sorry, but this is simply not true.
 
 
 The ONLY way to do that is if you do not care at all about user's data
 and simply accept that a rollback will also remove user data.
 
 The reason is simple: lot's of software *changes* data as part of its
 normal functioning, including and often in rollback-incompatible ways.

  What user data?  There is no user data touched/created during offline upgrade.

-- 
Tomasz TorczThere exists no separation between gods and men:
xmpp: zdzich...@chrome.pl   one blends softly casual into the other.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 25.01.2014 23:26, schrieb Tomasz Torcz:
 On Sat, Jan 25, 2014 at 02:55:32PM -0500, Simo Sorce wrote:
 The ONLY way to do that is if you do not care at all about user's data
 and simply accept that a rollback will also remove user data.

 The reason is simple: lot's of software *changes* data as part of its
 normal functioning, including and often in rollback-incompatible ways.
 
 What user data?  There is no user data touched/created during offline upgrade

and what is with the data *between* the upgrade and decision to roll back?



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

2014-01-25 Thread Adam Williamson

On Sat, 2014-01-25 at 23:26 +0100, Tomasz Torcz wrote:
 On Sat, Jan 25, 2014 at 02:55:32PM -0500, Simo Sorce wrote:
  On Sat, 2014-01-25 at 17:46 +0100, Tomasz Torcz wrote:

If there is a directory that contains update and non-update related file
changes, that's a problem. If there's segmentation, then this can be 
done.  
   
 Note that this situation is perfectly handled by Offline Updates.
   After reboot, there aren't collateral changes to filesystem, only 
   upgrade-related
   ones. So if there's a need for revert, the previous state is clearly 
   defined.
  
  Sorry, but this is simply not true.
  
  
  The ONLY way to do that is if you do not care at all about user's data
  and simply accept that a rollback will also remove user data.
  
  The reason is simple: lot's of software *changes* data as part of its
  normal functioning, including and often in rollback-incompatible ways.
 
   What user data?  There is no user data touched/created during offline 
 upgrade.

No, but you may have to use the system somewhat before you can find out
there was a problem with the upgrade, and at *that* point, your user
data may now be tied to the new versions of system apps, as Simo
describes.

So, it goes like this:

* Do an offline update that includes Foo v2.0
* Boot the updated system, run Foo, it migrates its configuration to
some new scheme
* Realize there was something wrong with the update, roll it back
* Run Foo again, find it doesn't work because it's been migrated to the
new config scheme which the old version of Foo doesn't work with
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350


On Jan 25, 2014, at 9:41 AM, Kevin Kofler kevin.kof...@chello.at wrote:

 Chris Murphy wrote:
 If there is a directory that contains update and non-update related file
 changes, that's a problem. If there's segmentation, then this can be done.
 
 Clearly /home needs to be separate (it's OK to take a snapshot but just
 don't use it by default in a rollback) or we lose changes in /home in a
 rollback from the time of the snapshot to the time of the decision to
 rollback.
 
 Another possible case it's /etc/ where the either a package or the user
 could make changes during the update.
 
 There's also /root, and then the most annoying case: /var. /var/lib/rpm 
 definitely needs to be rolled back, but you DON'T want to roll back things 
 such as log files in /var/log or systemwide databases (other than the RPM 
 database).

Well it might be woefully ignorant for me to say, and seem like flamebaiting, 
but the mixing of such domains tells me that the FHS needs revision even 
outside of the context of snapshots. It's just that snapshots makes it more 
obvious the organization is deficient.

Another weird one for me is /var/lib/libvirt/images which I certainly wouldn't 
want to snapshot (more specifically I wouldn't want to rollback by default in 
the face of a bad update).

Another way of dealing with this is many more subvolumes so that they can be 
selectively snapshotted for rollbacks while others remain persistent. Again 
it's fine to snapshot them at the same time also, but the rollback behavior by 
default would only rollback the software updates.

For point of comparison, when choosing the default Btrfs layout opensuse's 
installer creates three partitions: swap, root (btrfs), home (btrfs). And 
creates the following subvolumes on root:

boot/grub2/x86_64-efi
home
opt
srv
tmp
usr/local
var/crash
var/log
var/opt
var/spool
var/tmp

There's more snapshot granularity available with this setup.

Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350


On Jan 25, 2014, at 9:46 AM, Tomasz Torcz to...@pipebreaker.pl wrote:

 On Fri, Jan 24, 2014 at 03:10:04PM -0700, Chris Murphy wrote:
 
 Another possible case it's /etc/ where the either a package or the user could
 make changes during the update. Btrfs allows per file snapshots with cp
 --reflink so there might be a way to carve the snapshot with a scalpel but I
 prefer doing it with subvolume granularity. Plus that granularity translates 
 to
 LVM.  
 
  Note that this situation is perfectly handled by Offline Updates.
 After reboot, there aren't collateral changes to filesystem, only 
 upgrade-related
 ones. So if there's a need for revert, the previous state is clearly defined.

I don't follow this. The realization an update is bad doesn't necessarily occur 
right away. So we still need a way to separate system domain vs user domain, at 
least, so that system files are rolled back separately from user files.


Chris Murphy

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 25, 2014, at 12:55 PM, Simo Sorce s...@redhat.com wrote:

 The reason is simple: lot's of software *changes* data as part of its
 normal functioning, including and often in rollback-incompatible ways.
 
 You cannot assume that upgrading a program that uses a database X from
 version A to B can still work if you keep database X unchanged and then
 rollback from B to A. Lot of applications apply changes to database at
 upgrade time, either in the rpm scriplets or automatically as soon as a
 new version binary is run.

I wouldn't ever expect this with a minor version or security update. I'd 
consider it a complete betrayal of software versioning to do this. In fact in 
certain instances of major version changes, downward compatibility of the file 
format is expected.

 It is basically impossible to find applications that handle the case
 where you downgrade, in any more graceful way than punting and failing
 to start in the *good* case. In the bad case they start and trash the
 database.

I guess I'm not really following. Do you have a for instance? Because off hand 
this sounds like a kind of sabotage to me. If it's throw away database info 
that can simply be reconstructed I'd probably tolerate it. But for that matter, 
such things should go in an defined cache location so that it's not even being 
backed up.

But important user data having it's format updated in a way that makes it 
incompatible with the previous minor version (same major version)? I'm 
snickering at the language that would ensue in the proprietary software world, 
if I'm not totally confused about what it is you're suggested is fair game. 
It'd be the sort of language you wouldn't want your teenager or grandmother to 
hear.


Chris Murphy
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]

Am 26.01.2014 01:28, schrieb Chris Murphy:
 It is basically impossible to find applications that handle the case
 where you downgrade, in any more graceful way than punting and failing
 to start in the *good* case. In the bad case they start and trash the
 database.

 But important user data having it's format updated in a way that makes it 
 incompatible
 with the previous minor version (same major version)? I'm snickering at the 
 language that 
 would ensue in the proprietary software world

you do not know what happens in case *of a bug*
you are in the area of undefined behavior

the point is that the snapshot *does not* bring you for
sure back where you came from or if it does you may regret
it because there is a timewindow between 3 steps

* snapshot / update
* continue your normal operations
* recognize there is a problem
* restore the snapshot
_

* if i have /var on a seperate partition *god beware* of the idea
  rollback a snapshot of the remaining rootfs because the system
  is ruined - /var contains the whole rpm-database

* if i have /home on the same FS as the rootfs - *god beware* of restore
  a snapshot because all work before recognize there is a problem is
  ruined
_

well, people already statet the solution maybe split the OS granulary
and extend the FHS - that will *not* solve the problem, it only will
create new ones beause at the end of the day nobody except very few
people know what is hwre stored, snapshotted and can be restored with
what exactly impact leading to lose any control

a bug is a bug and in case of undefined bahvior the word *undefined*
is the really important - frankly, what happens if one of the components
used for snapshots is affected?

* nothing
* undefined system state
* all get trashed

solving problems by add more layers of complexity leads to have more
layers prone for bugs themself and the IT after 2010 tends to solve
that by wrap another layer around. frustrating.

Linux would not exist if Unix would not have made it a different way and
people coming up with technical complex solutions should consider how
it can be that 30 or 40 years old solutions still working perfect and
all the new ones are replaced every 2-5 years



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]


On Jan 25, 2014, at 4:12 PM, Adam Williamson awill...@redhat.com wrote:
 
 * Do an offline update that includes Foo v2.0
 * Boot the updated system, run Foo, it migrates its configuration to
 some new scheme
 * Realize there was something wrong with the update, roll it back
 * Run Foo again, find it doesn't work because it's been migrated to the
 new config scheme which the old version of Foo doesn't work with

I would grumble, but a configuration file being updated and made incompatible 
with the prior version would be tolerated. Ideally the application makes an 
unmodified copy. If it doesn't, new school restore with --reflink from 
snapshot, regular cp if using LVM thinp snapshots, and old school just restore 
the file from a conventional backup. Not such a big deal.

If it's something far less throw away than configuration files being changed, 
it's a bit more complicated how badly and quickly the conversation degrades. 
But I can hardly recall a recent example of this happening. It's just not that 
common in my experience.


Chris Murphy
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Snapshotting for rollback after updates was[ Re: Drawing lessons from fatal SELinux bug #1054350]



Am 26.01.2014 01:54, schrieb Chris Murphy:
 On Jan 25, 2014, at 4:12 PM, Adam Williamson awill...@redhat.com wrote:

 * Do an offline update that includes Foo v2.0
 * Boot the updated system, run Foo, it migrates its configuration to
 some new scheme
 * Realize there was something wrong with the update, roll it back
 * Run Foo again, find it doesn't work because it's been migrated to the
 new config scheme which the old version of Foo doesn't work with
 
 I would grumble, but a configuration file being updated and made incompatible 
 with the prior version would be tolerated. Ideally the application makes an 
 unmodified copy. If it doesn't, new school restore with --reflink from 
 snapshot, regular cp if using LVM thinp snapshots, and old school just 
 restore the file from a conventional backup. Not such a big deal

the short version of ahwat you said could have been forget snapshots at all to 
solve
such problems to not lead dvelopers into temptation of i can be less caeful 
because
we have snapshots

in other words: don't work around problems by create new ones   




signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Sergio Pascual

2014/1/24 Ralf Corsepius rc040...@freenet.de


 Certainly, downgrading installations which already upgraded to faulty
 packages would not work.

 Ralf


The situation (a broken system that cannot be upgraded)  could be mitigated
a little bit by using yum + system snapshots. You can rollback to a
previous sane system.

There is a plugin yum-plugin-fs-snapshot, but it requires better
documentation and system integration.

Currently (I don't know how current is F16 documentation) it requires
running lvm by hand

http://docs.fedoraproject.org/en-US/Fedora/16/html/System_Administrators_Guide/sec-Plugin_Descriptions.html

Sergio



 --
 devel mailing list
 devel@lists.fedoraproject.org
 https://admin.fedoraproject.org/mailman/listinfo/devel
 Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread drago01

On Fri, Jan 24, 2014 at 12:55 AM, Kevin Kofler kevin.kof...@chello.at wrote:


 So, what happened:
 * We are enabling SELinux enabled (enforcing) by default, a tool designed to
 prevent anything it does not like from happening. (Reread this carefully:
 The ONLY thing that tool is designed to do at all is PREVENT things. It does
 not have a SINGLE feature other than being a roadblock and an annoyance.)

The feature is called security. By your logic everyone should be
root, we should
disable other security features like ASLR and NX (both PREVENT me from running
malicious code but do not add a SINGLE feature).

So please read on how security is implemented and why.

 * SELinux works by shipping a policy that effectively tries to specify in
 one single place (read: single point of failure!) everything any program in
 Fedora (scalability disaster!) ever wants to do (second-guessing its actual
 code, i.e., duplication of all logic!).

That's not how it works not how it supposed to work. Please read on MAC.

 (Note the 3 (!) major antipatterns
 in a single-sentence (!) description of how SELinux works!)

Not a description on how it works but your misunderstand.

 * An update to that SELinux policy was shipped that BREAKS the most critical
 tools in Fedora, the ones required to update the system and thus install the
 fixes for any regressions, including the very regression that caused the
 breakage. And also any automated workarounds are blocked by design.

No idea what automated workaround means but there are other ways to
deal with it see Colin's post.

 * That update made it out to the stable updates! In other words, the
 draconian Update Policies that were enacted in a vain attempt to prevent
 such issues from happening utterly failed at catching this bug.

Yeah so we should find out why this happened and improve the testing
procedures to not let it happen in the feature (again see Colin's mail).


 So, what needs to happen:
 * SELinux must be disabled (or preferably, not installed in the first place,
 to avoid wasting space for nothing) by default! Just consider the benefits
 (none!)

As stated above that's not true.

 * The Update Policies must be repealed. This regression has shown us that
 not only they totally failed at preventing it, but they are actively
 contributing to exposing MORE users to broken updates by delaying regression
 fixes. (This kind of regression fixes needs to go out DIRECTLY to stable!)

This is a contradiction our current testing didn't find the bug so
how about we do no testing at all.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Kevin Kofler

Adam Williamson wrote:
 Even if we can do it on the mirrors, we have no way to 'recall' a
 package from systems where it's already been installed (of course in the
 current case that wouldn't have worked anyway, but we're discussing the
 generic case here).

Crazy idea of the day: Maybe our update tools should default to distro-sync 
rather than update? Together with ensuring timestamp monotonicity on the 
metadata (don't accept older metadata if you already have newer one), it 
would allow easily pulling faulty updates (except when RPM is broken as in 
this case, of course) and could even render the dreaded Epoch hack obsolete.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Kevin Kofler

Adam Williamson wrote:
 TBH this has always been the one of Kevin's Big Book Of Update Policy
 Complaints I find the most baffling. If we know you managed to screw up
 your update once, why exactly would we just trust you to get it right
 the *second* time without any testing?

* If the package is already so screwed that it breaks the whole system, it 
cannot realistically get any worse.
* A regression fix is usually a trivial change, often reverting something to 
a previous, already well-tested, state.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Kevin Kofler

drago01 wrote:
 The feature is called security. By your logic everyone should be
 root,

For home user machines, that wouldn't necessarily be a bad thing (but it 
would mean fixing the software that special-cases the root user improperly 
for no good reason).

Alternatively, the kernel could be patched to give admin users (either 
defined as members of the wheel group as now, or by some additional 
property that would be set for the same users by default) some strategic 
capabilities such as dac_override. That would also put an end to the endless 
annoyance of having to sudo all the time. (And by the way, sudo and 
PolicyKit actions should be allowed with no password (rather than the user 
password as now) for wheel group members by default.) That way, you still 
get the benefits from different accounts, e.g., different preferences per 
family member, without the current restrictions imposed to normal users.

The endless password prompts make a lot of sense in controlled corporate 
environments with dedicated system administrators, but on home machines, 
they are just an unnecessary annoyance.

 * SELinux works by shipping a policy that effectively tries to specify
 in one single place (read: single point of failure!) everything any
 program in Fedora (scalability disaster!) ever wants to do
 (second-guessing its actual code, i.e., duplication of all logic!).
 
 That's not how it works not how it supposed to work. Please read on MAC.

Uh, I know how it works. The above is how I summarize it. If you think that 
is incorrect, please explain HOW.

 * An update to that SELinux policy was shipped that BREAKS the most
 critical tools in Fedora, the ones required to update the system and thus
 install the fixes for any regressions, including the very regression that
 caused the breakage. And also any automated workarounds are blocked by
 design.
 
 No idea what automated workaround means but there are other ways to
 deal with it see Colin's post.

A %pretrans scriptlet that fixes the problem without manual user 
intervention (other than OKing the update). But SELinux won't allow RPMs to 
mess with it that way (especially without invoking an external executable, 
which is blocked by the faulty policy) because it would defeat its flawed 
security model.

 Yeah so we should find out why this happened and improve the testing
 procedures to not let it happen in the feature (again see Colin's mail).

NO amount of testing is going to prevent regressions from happening 
occasionally. This means:
* we need to eliminate common sources of regressions such as SELinux, to 
prevent whole classes of regressions from occurring in the first place 
(prevention is better than duct tape!) and
* we have to accept that regressions can always happen and allow for fast 
fixes to those regressions (direct stable pushes).

 So, what needs to happen:
 * SELinux must be disabled (or preferably, not installed in the first
 place, to avoid wasting space for nothing) by default! Just consider the
 benefits (none!)
 
 As stated above that's not true.

As stated above, that IS true. :-)

 * The Update Policies must be repealed. This regression has shown us that
 not only they totally failed at preventing it, but they are actively
 contributing to exposing MORE users to broken updates by delaying
 regression fixes. (This kind of regression fixes needs to go out DIRECTLY
 to stable!)
 
 This is a contradiction our current testing didn't find the bug so
 how about we do no testing at all.

There is no contradiction. Doing away with policies that do not work is 
perfectly logical, as is allowing quick regression fixes because regressions 
do happen no matter how much you test.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350


Am 24.01.2014 13:56, schrieb Kevin Kofler:
 Alternatively, the kernel could be patched to give admin users (either 
 defined as members of the wheel group as now, or by some additional 
 property that would be set for the same users by default) some strategic 
 capabilities such as dac_override. That would also put an end to the endless 
 annoyance of having to sudo all the time. (And by the way, sudo and 
 PolicyKit actions should be allowed with no password (rather than the user 
 password as now) for wheel group members by default.) That way, you still 
 get the benefits from different accounts, e.g., different preferences per 
 family member, without the current restrictions imposed to normal users.
 
 The endless password prompts make a lot of sense in controlled corporate 
 environments with dedicated system administrators, but on home machines, 
 they are just an unnecessary annoyance

no, they are not, they have the same reason as firefox asks
for the master-password before display stored passwords even
after you already entered it to login somewhere

they prevent that if you are not alone that while you go to
the toilet and forget to lock your screen unauthorized people
not doing things nobody wants on the machine

what you propose is the Apple way - not on a linux system please




signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Simo Sorce

On Fri, 2014-01-24 at 14:40 +0100, Reindl Harald wrote:
 Am 24.01.2014 13:56, schrieb Kevin Kofler:
  Alternatively, the kernel could be patched to give admin users (either 
  defined as members of the wheel group as now, or by some additional 
  property that would be set for the same users by default) some strategic 
  capabilities such as dac_override. That would also put an end to the 
  endless 
  annoyance of having to sudo all the time. (And by the way, sudo and 
  PolicyKit actions should be allowed with no password (rather than the user 
  password as now) for wheel group members by default.) That way, you still 
  get the benefits from different accounts, e.g., different preferences per 
  family member, without the current restrictions imposed to normal users.
  
  The endless password prompts make a lot of sense in controlled corporate 
  environments with dedicated system administrators, but on home machines, 
  they are just an unnecessary annoyance
 
 no, they are not, they have the same reason as firefox asks
 for the master-password before display stored passwords even
 after you already entered it to login somewhere
 
 they prevent that if you are not alone that while you go to
 the toilet and forget to lock your screen unauthorized people
 not doing things nobody wants on the machine

Worse than that, they prevent automated attacks via very vulnerable
applications like browsers. [which of course in Kevin's world are never
run in a SELinux sandbox]

So you if you get some malware to jailbreak out of the browser sandbox
all it needs to do is sudo pwnme if there is no password request.

Of course you need to understand at least a smidget of security to avoid
proposing ludicrous 'defaults'.

 what you propose is the Apple way - not on a linux system please

It is just 'the pwn me' way, nothing more, nothing less.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Ralf Corsepius


On 01/24/2014 01:39 PM, Kevin Kofler wrote:

Adam Williamson wrote:

Even if we can do it on the mirrors, we have no way to 'recall' a
package from systems where it's already been installed (of course in the
current case that wouldn't have worked anyway, but we're discussing the
generic case here).


Crazy idea of the day: Maybe our update tools should default to distro-sync
rather than update?

No, for 2 reasons:

a) This would blow away all installed packages, which aren't available 
in permanently enabled repos.
 Most common such case is having selectively installed packages from 
updates-testing, because users are facing problems with these packages' 
nominal versions.


b) A much more common packaging bug class than the SELinux-case are 
packages, which can not be uninstalled or downgraded or not be 
downgraded properly. Classic such cases are packages with defective 
rpm-scriptlets or with scriptlet which perform persistent changes.


Ralf

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

Am 24.01.2014 15:55, schrieb Ralf Corsepius:
 On 01/24/2014 01:39 PM, Kevin Kofler wrote:
 Adam Williamson wrote:
 Even if we can do it on the mirrors, we have no way to 'recall' a
 package from systems where it's already been installed (of course in the
 current case that wouldn't have worked anyway, but we're discussing the
 generic case here).

 Crazy idea of the day: Maybe our update tools should default to distro-sync
 rather than update?
 No, for 2 reasons:
 
 a) This would blow away all installed packages, which aren't available in 
 permanently enabled repos

that is not true, try it out

otherwise some packages would be not installed on my machines after a 
dist-upgrade
namely the ones never came from any repo and installed locally

 Most common such case is having selectively installed packages from 
 updates-testing, because users are facing
 problems with these packages' nominal versions

*that* is the reason not to do so because it would downgrade anything updated
explicitly from updates-testing,kde-testing,koji which would be a bad default



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Ralf Corsepius


On 01/24/2014 04:06 PM, Reindl Harald wrote:

Am 24.01.2014 15:55, schrieb Ralf Corsepius:

On 01/24/2014 01:39 PM, Kevin Kofler wrote:

Adam Williamson wrote:

Even if we can do it on the mirrors, we have no way to 'recall' a
package from systems where it's already been installed (of course in the
current case that wouldn't have worked anyway, but we're discussing the
generic case here).


Crazy idea of the day: Maybe our update tools should default to distro-sync
rather than update?

No, for 2 reasons:

a) This would blow away all installed packages, which aren't available in 
permanently enabled repos


that is not true, try it out


Been there many times.


Real world example with a package I maintain, which currently has an 
update pending in updates-testing:



# yum install gumbo-parser
...
Installing : gumbo-parser-1.0-0.2.20131001gitd90ea2b.fc20.x86_64
...
[Note: updates-testing is disabled in 
/etc/yum.repo.d/fedora-updates-testing.repo]



Now temporarily enable updates-testing to pull in the package from 
updates-testing for testing:

# yum update --enablerepo=updates-testing gumbo-parser
...
Updating   : gumbo-parser-1.0-0.2.20131204git87b99f2.fc20.x86_64
...


# yum distro-sync
...
Downgrading:
gumbo-parser  x86_64 
  1.0-0.2.20131001gitd90ea2b.fc20   fedora

...
Removed:
  gumbo-parser.x86_64 0:1.0-0.2.20131204git87b99f2.fc20 




Installed:
  gumbo-parser.x86_64 0:1.0-0.2.20131001gitd90ea2b.fc20
...
=

qed


Ralf

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350



Am 24.01.2014 16:40, schrieb Ralf Corsepius:
 On 01/24/2014 04:06 PM, Reindl Harald wrote:
 a) This would blow away all installed packages, which aren't available in 
 permanently enabled repos

 that is not true, try it out
 Been there many times

no, you did not and you did also not in your example below

 Real world example with a package I maintain, which currently has an update 
 pending in updates-testing:

 # yum distro-sync
 ...
 Downgrading:
 gumbo-parser  x86_64  
 1.0-0.2.20131001gitd90ea2b.fc20   fedora
 ...
 Removed:
   gumbo-parser.x86_64 0:1.0-0.2.20131204git87b99f2.fc20
 
 Installed:
   gumbo-parser.x86_64 0:1.0-0.2.20131001gitd90ea2b.fc20

nothing is blown away, you only did not read the output
because it was *downgraded* and *not* removed

this is *completly* different than blown away
this is what distro-sync *is supposed to do*
upgrade or downgrade any package which is in whatever current repo
but it *does not* blow away packages not in any repo at all


and if you would not have stripped this paragraph of my original
reply you maybe had looked twice

 *that* is the reason not to do so because it would downgrade anything updated
 explicitly from updates-testing,kde-testing,koji which would be a bad default



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Ralf Corsepius


On 01/24/2014 04:57 PM, Reindl Harald wrote:



Am 24.01.2014 16:40, schrieb Ralf Corsepius:

On 01/24/2014 04:06 PM, Reindl Harald wrote:

a) This would blow away all installed packages, which aren't available in 
permanently enabled repos


that is not true, try it out

Been there many times


no, you did not and you did also not in your example below


Real world example with a package I maintain, which currently has an update 
pending in updates-testing:

# yum distro-sync
...
Downgrading:
gumbo-parser  x86_64
1.0-0.2.20131001gitd90ea2b.fc20   fedora
...
Removed:
   gumbo-parser.x86_64 0:1.0-0.2.20131204git87b99f2.fc20

Installed:
   gumbo-parser.x86_64 0:1.0-0.2.20131001gitd90ea2b.fc20


nothing is blown away, you only did not read the output
because it was *downgraded* and *not* removed

Rubbish - Stop being childish.


this is *completly* different than blown away
this is what distro-sync *is supposed to do*
upgrade or downgrade any package which is in whatever current repo
but it *does not* blow away packages not in any repo at all
It if the package from updates-testing was fixing a critical bug on your 
system, your system would be malfunctioning afterwards.


--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350



Am 24.01.2014 17:12, schrieb Ralf Corsepius:
 On 01/24/2014 04:57 PM, Reindl Harald wrote:
 Am 24.01.2014 16:40, schrieb Ralf Corsepius:
 On 01/24/2014 04:06 PM, Reindl Harald wrote:
 a) This would blow away all installed packages, which aren't available in 
 permanently enabled repos

 that is not true, try it out
 Been there many times

 no, you did not and you did also not in your example below

 Real world example with a package I maintain, which currently has an update 
 pending in updates-testing:

 # yum distro-sync
 ...
 Downgrading:
 gumbo-parser  x86_64
 1.0-0.2.20131001gitd90ea2b.fc20   fedora
 ...
 Removed:
gumbo-parser.x86_64 0:1.0-0.2.20131204git87b99f2.fc20

 Installed:
gumbo-parser.x86_64 0:1.0-0.2.20131001gitd90ea2b.fc20

 nothing is blown away, you only did not read the output
 because it was *downgraded* and *not* removed
 Rubbish - Stop being childish.

nobody here is childish, except maybe you

 this is *completly* different than blown away
 this is what distro-sync *is supposed to do*
 upgrade or downgrade any package which is in whatever current repo
 but it *does not* blow away packages not in any repo at all

 It if the package from updates-testing was fixing a critical bug on your 
 system, your system would be
 malfunctioning afterwards

and exactly *that* was what i said in my first reply while you
stripped *exactly* that part out from your quote, most likely
because you replied with a reflex without read exactly 5 lines
completly

but that is *not* a) This would blow away all installed packages, which aren't 
available in
permanently enabled repos because that would mean *uninstall* any package 
which is currently
not in a enabled repo - and that is *not* what distro-sync does

below *again* my complete reply which is and was technical correct
while your would blow away is not

so before call others childish the next time before you reply to a message
read also the second pararaph to avoid useless discussions

 Original-Nachricht 
Betreff: Re: Drawing lessons from fatal SELinux bug #1054350
Datum: Fri, 24 Jan 2014 16:06:21 +0100
Von: Reindl Harald h.rei...@thelounge.net
An: devel@lists.fedoraproject.org

Am 24.01.2014 15:55, schrieb Ralf Corsepius:
 On 01/24/2014 01:39 PM, Kevin Kofler wrote:
 Adam Williamson wrote:
 Even if we can do it on the mirrors, we have no way to 'recall' a
 package from systems where it's already been installed (of course in the
 current case that wouldn't have worked anyway, but we're discussing the
 generic case here).

 Crazy idea of the day: Maybe our update tools should default to distro-sync
 rather than update?
 No, for 2 reasons:

 a) This would blow away all installed packages, which aren't available in 
 permanently enabled repos

that is not true, try it out

otherwise some packages would be not installed on my machines after a 
dist-upgrade
namely the ones never came from any repo and installed locally

 Most common such case is having selectively installed packages from 
 updates-testing, because users are facing
 problems with these packages' nominal versions

*that* is the reason not to do so because it would downgrade anything updated
explicitly from updates-testing,kde-testing,koji which would be a bad default



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Adam Williamson

On Fri, 2014-01-24 at 10:58 +0100, Sergio Pascual wrote:
 2014/1/24 Ralf Corsepius rc040...@freenet.de
 
 Certainly, downgrading installations which already upgraded to
 faulty packages would not work.
 
 Ralf
 
 
 
 The situation (a broken system that cannot be upgraded)  could be
 mitigated a little bit by using yum + system snapshots. You can
 rollback to a previous sane system.
 
 There is a plugin yum-plugin-fs-snapshot, but it requires better
 documentation and system integration.
 
 
 Currently (I don't know how current is F16 documentation) it requires
 running lvm by hand 
 
 http://docs.fedoraproject.org/en-US/Fedora/16/html/System_Administrators_Guide/sec-Plugin_Descriptions.html

AIUI there is/was a long-term plan to integrate this as core
functionality using btrfs snapshots - in fact that was one of the major
attractions of the idea of switching to btrfs-by-default in the first
place. I believe those involved didn't think the LVM-based
implementation was clean/robust enough to use by default, but a
btrfs-based implementation would be. Do correct me if I'm wrong.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Adam Williamson

On Fri, 2014-01-24 at 13:36 +0100, Kevin Kofler wrote:
 Adam Williamson wrote:
  TBH this has always been the one of Kevin's Big Book Of Update Policy
  Complaints I find the most baffling. If we know you managed to screw up
  your update once, why exactly would we just trust you to get it right
  the *second* time without any testing?
 
 * If the package is already so screwed that it breaks the whole system, it 
 cannot realistically get any worse.

Sure it can. It can wipe all your data, or mail it to the NSA...

Breaks the whole system is high on the Pantscon Scale, sure, but it's
not the highest. Data loss and security compromise both come higher.

 * A regression fix is usually a trivial change, often reverting something to 
 a previous, already well-tested, state.

Sure. And what could possibly go wrong.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Fabian Deutsch

Am Freitag, den 24.01.2014, 00:55 +0100 schrieb Kevin Kofler:
 it is time to analyze the fallout from the following catastrophic
 Fedora 20 
 regression:
 https://bugzilla.redhat.com/show_bug.cgi?id=1054350
 rpm scriptlets are exiting with status 127

Hey,

can't we add a default boot entry which starts the system in permissive
mode?

- fabian

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread drago01

On Fri, Jan 24, 2014 at 7:12 PM, Fabian Deutsch fabian.deut...@gmx.de wrote:
 Am Freitag, den 24.01.2014, 00:55 +0100 schrieb Kevin Kofler:
 it is time to analyze the fallout from the following catastrophic
 Fedora 20
 regression:
 https://bugzilla.redhat.com/show_bug.cgi?id=1054350
 rpm scriptlets are exiting with status 127

 Hey,

 can't we add a default boot entry which starts the system in permissive
 mode?

How would that help? If a user knows enough about the issue to try it
he/she could just switch to permissive mode.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Kevin Fenzi

On Fri, 24 Jan 2014 09:41:13 -0800
Adam Williamson awill...@redhat.com wrote:

 AIUI there is/was a long-term plan to integrate this as core
 functionality using btrfs snapshots - in fact that was one of the
 major attractions of the idea of switching to btrfs-by-default in the
 first place. I believe those involved didn't think the LVM-based
 implementation was clean/robust enough to use by default, but a
 btrfs-based implementation would be. Do correct me if I'm wrong.

I don't think snapshots are a partcularly good solution, unless there's
some way to only roll back the rpm/yum transaction without also rolling
back unrelated changes. 

kevin


signature.asc
Description: PGP signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350

2014-01-24 Thread Michael Schwendt

On Fri, 24 Jan 2014 00:55:23 +0100, Kevin Kofler wrote:

 it is time to analyze the fallout from the following catastrophic Fedora 20 
 regression:
 https://bugzilla.redhat.com/show_bug.cgi?id=1054350
 rpm scriptlets are exiting with status 127

 * We are losing users to Ubuntu because of this issue.

Always hard to comment on without have seen numbers/statistics first.

IMO, there's some sort of pork cycle related to users switching
distributions forth and back after a couple of releases. Around Fedora 18
users of Ubuntu have returned. The current Anaconda is not everyone's
coup of tea, however, so that's one point where users are lost already.

 * The bug now has 38 (!) duplicates in Bugzilla, plus many complaints on 
 IRC, mailing lists, comments to other unrelated bugs (the fix for which 
 cannot be installed due to the SELinux bug) etc.

As I've searched for and closed several of the dupes, the various bug
reports are interesting. The users have misidentified the problem as
being a problem in the package/update they wanted to install. Some have
noticed many packages failing and have blamed Yum. I wonder how many
users are affected by the problem and have not done anything yet (since,
for example, they expect an update to fix it magically).

 * That update made it out to the stable updates! In other words, the 
 draconian Update Policies that were enacted in a vain attempt to prevent 
 such issues from happening utterly failed at catching this bug.

Those policies are not draconian enough [1]. On erroneous belief that
a +1 from three different testers would mean that the update has seen
enough testing, the test update has been published with the default karma
threshold of +3. The testers have failed. It's too simple for testers to
rush through the voting in bodhi without testing the updates
painstakingly. The faster the better has lead to a fatal mistake in
this case.

[1] It's up to the package maintainers to disable karma automatism or
to increase the threshold. AFAIK, the selinux maintainers are open to
doing exactly that.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350


Am 24.01.2014 19:18, schrieb drago01:
 On Fri, Jan 24, 2014 at 7:12 PM, Fabian Deutsch fabian.deut...@gmx.de wrote:
 Am Freitag, den 24.01.2014, 00:55 +0100 schrieb Kevin Kofler:
 it is time to analyze the fallout from the following catastrophic
 Fedora 20
 regression:
 https://bugzilla.redhat.com/show_bug.cgi?id=1054350
 rpm scriptlets are exiting with status 127

 Hey,

 can't we add a default boot entry which starts the system in permissive
 mode?
 
 How would that help? If a user knows enough about the issue to try it
 he/she could just switch to permissive mode

in *that* case

in a case where a broken selinux update leads in not boot at all
i can not imagine what i would to besides boot with a CD/DVD/USB



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Drawing lessons from fatal SELinux bug #1054350