Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Josef Bacik]

On Wed, Oct 12, 2016 at 8:35 PM, Gerald B. Cox  wrote:
>
> On Wed, Oct 12, 2016 at 3:29 PM, Chris Murphy 
> wrote:
>>
>> On Wed, Oct 12, 2016 at 6:29 AM, Gerald B. Cox  wrote:
>> > On Tue, Oct 11, 2016 at 8:52 PM, Chris Murphy 
>> > wrote:
>> >>
>> >>
>> >> About the rewrite comment: that did not come from a developer, and is
>> >> definitely overstated. In any case, rewrites are not inherently bad
>> >> news, there's a bunch of OpenZFS videos from last yearss summit in
>> >> which the developers talk about various things being completely
>> >> rewritten from scratch, some things more than twice. So kinda par for
>> >> the course, and given enough time things get rewritten anyway. XFS has
>> >> had substantial changes over its history including numerous on disk
>> >> format changes even before it found its way onto Linux.
>> >
>> >
>> > Could be, should be, may be... that's fine - but it all says the same
>> > thing... they
>> > don't know how much time it is going to take to fix - and who knows what
>> > their
>> > priority is to get around to it.  The advantages over what already is
>> > available
>> > don't appear to be that compelling, especially when weighed with the
>> > risks.
>>
>> So you are saying that you started using raid56 when it was brand new,
>> before it had *any* kind of persistent repairing or device replacement
>> and only now, due to a bug that manifests remarkably less bad than the
>> normal behavior of everything else (minus ZFS), are you now
>> complaining? So basically this is, "I want it now!" complaint. Because
>> all the available information has been saying don't use raid56 for
>> production, but you did so anyway. This is a subjective change in your
>> evaluation. It has nothing to do with the state of Btrfs so you really
>> shouldn't blame it when your requirements have clearly changed.
>
>
> Well no, what I said was that I thought that they might have a somewhat
> stable product
> after three years... instead I get that a rewrite may be required with no
> idea of when it would ever
> be production ready.  If you think that is acceptable, more power to you.  I
> do not and I would
> venture to guess that many reasonable people would think three years would
> be ample time.
> My requirements haven't changed... I just think the time to fish or cut bait
> has come.  I've cut
> bait.
>
>>
>>
>> >
>> > When all this started I did some searches and found Kent Overstreet's
>> > page
>> > on
>
>
>>
>> > bcachefs:  https://goo.gl/U0UFfN
>>
>>
>> >
>> > He had some words about the different filesystems - and had this to say
>> > about btrfs:
>> >
>> > btrfs, which was supposed to be Linux's next generation COW filesystem -
>> > Linux's answer to zfs. Unfortunately, too much code was written too
>> > quickly
>> > without focusing on getting the core design correct first, and now it
>> > has
>> > too many design mistakes baked into the on disk format and an enormous,
>> > messy codebase - bigger that xfs. It's taken far too long to stabilize
>> > as
>> > well - poisoning the well for future filesystems because too many people
>> > were burned on btrfs, repeatedly (e.g. Fedora's tried to switch to btrfs
>> > multiple times and had to switch at the last minute, and server vendors
>> > who
>> > years ago hoped to one day roll out btrfs are now quietly migrating to
>> > xfs
>> > instead).
>>
>> I have heard from a couple developers that it was a victim of its own
>> hype/success and too many feature additions without equivalent effort
>> on error reporting, debugging, and fault injection tools. I have yet
>> to hear a Btrfs developer say the core design or on disk format has
>> anything to do with the problems, but to the contrary. The comment
>> it's bigger than XFS is kinda funny, seeing as it does more than XFS,
>> md, and LVM combined, so a proper comparison would be comparing all of
>> them to Btrfs minus their user space tools (for sure Btrfs tools do
>> not come anywhere near the metric ass tonne of switches or
>> documentation in LVM or mdadm).
>
>
> I can't speak for him, but don't believe that was his point.  I read it as
> that the
> code was bloated.
>>
>>
>> The Fedora report is simply nonsense. Fedora made no meaningful
>> attempt to switch to Btrfs once, let alone multiple times. FESCo
>> considered and approved it, with conditions attached.
>
>
> Well, we're getting into semantics here.  I would call that a serious
> attempt - and there
> are many articles that are available that talk about Fedora discussing
> making BTRFS
> a default.  If you don't consider any of those attempts "meaningful" I
> suppose that's
> a good thing.
>
>>
>> Josef kept
>> pushing it off because he thought it wasn't ready.
>
>
> Smart man, he was right.
>
>>
>> And then Josef
>> moved on from Red Hat, and wasn't replaced. Characterizing this as
>> "tried to switch" and "had to switch at the last minute" is at best
>> 

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Wed, Oct 12, 2016 at 3:29 PM, Chris Murphy 
wrote:

> On Wed, Oct 12, 2016 at 6:29 AM, Gerald B. Cox  wrote:
> > On Tue, Oct 11, 2016 at 8:52 PM, Chris Murphy 
> > wrote:
> >>
> >>
> >> About the rewrite comment: that did not come from a developer, and is
> >> definitely overstated. In any case, rewrites are not inherently bad
> >> news, there's a bunch of OpenZFS videos from last yearss summit in
> >> which the developers talk about various things being completely
> >> rewritten from scratch, some things more than twice. So kinda par for
> >> the course, and given enough time things get rewritten anyway. XFS has
> >> had substantial changes over its history including numerous on disk
> >> format changes even before it found its way onto Linux.
> >
> >
> > Could be, should be, may be... that's fine - but it all says the same
> > thing... they
> > don't know how much time it is going to take to fix - and who knows what
> > their
> > priority is to get around to it.  The advantages over what already is
> > available
> > don't appear to be that compelling, especially when weighed with the
> risks.
>
> So you are saying that you started using raid56 when it was brand new,
> before it had *any* kind of persistent repairing or device replacement
> and only now, due to a bug that manifests remarkably less bad than the
> normal behavior of everything else (minus ZFS), are you now
> complaining? So basically this is, "I want it now!" complaint. Because
> all the available information has been saying don't use raid56 for
> production, but you did so anyway. This is a subjective change in your
> evaluation. It has nothing to do with the state of Btrfs so you really
> shouldn't blame it when your requirements have clearly changed.
>

Well no, what I said was that I thought that they might have a somewhat
stable product
after three years... instead I get that a rewrite may be required with no
idea of when it would ever
be production ready.  If you think that is acceptable, more power to you.
I do not and I would
venture to guess that many reasonable people would think three years would
be ample time.
My requirements haven't changed... I just think the time to fish or cut
bait has come.  I've cut
bait.


>
> >
> > When all this started I did some searches and found Kent Overstreet's
> page
> > on
>


> > bcachefs:  https://goo.gl/U0UFfN
>
>
> >
> > He had some words about the different filesystems - and had this to say
> > about btrfs:
> >
> > btrfs, which was supposed to be Linux's next generation COW filesystem -
> > Linux's answer to zfs. Unfortunately, too much code was written too
> quickly
> > without focusing on getting the core design correct first, and now it has
> > too many design mistakes baked into the on disk format and an enormous,
> > messy codebase - bigger that xfs. It's taken far too long to stabilize as
> > well - poisoning the well for future filesystems because too many people
> > were burned on btrfs, repeatedly (e.g. Fedora's tried to switch to btrfs
> > multiple times and had to switch at the last minute, and server vendors
> who
> > years ago hoped to one day roll out btrfs are now quietly migrating to
> xfs
> > instead).
>
> I have heard from a couple developers that it was a victim of its own
> hype/success and too many feature additions without equivalent effort
> on error reporting, debugging, and fault injection tools. I have yet
> to hear a Btrfs developer say the core design or on disk format has
> anything to do with the problems, but to the contrary. The comment
> it's bigger than XFS is kinda funny, seeing as it does more than XFS,
> md, and LVM combined, so a proper comparison would be comparing all of
> them to Btrfs minus their user space tools (for sure Btrfs tools do
> not come anywhere near the metric ass tonne of switches or
> documentation in LVM or mdadm).
>

I can't speak for him, but don't believe that was his point.  I read it as
that the
code was bloated.

>
> The Fedora report is simply nonsense. Fedora made no meaningful
> attempt to switch to Btrfs once, let alone multiple times. FESCo
> considered and approved it, with conditions attached.


Well, we're getting into semantics here.  I would call that a serious
attempt - and there
are many articles that are available that talk about Fedora discussing
making BTRFS
a default.  If you don't consider any of those attempts "meaningful" I
suppose that's
a good thing.


> Josef kept
> pushing it off because he thought it wasn't ready.


Smart man, he was right.


> And then Josef
> moved on from Red Hat, and wasn't replaced. Characterizing this as
> "tried to switch" and "had to switch at the last minute" is at best
> hyperbole.


Ok, if you say so.  Again, that wasn't the way the media reported it.


> It was a change proposal, and it never met the requirements
> of either the proposer or FESCo for it to proceed further. No changes
> in default happened in 

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Wed, Oct 12, 2016 at 6:29 AM, Gerald B. Cox  wrote:
> On Tue, Oct 11, 2016 at 8:52 PM, Chris Murphy 
> wrote:
>>
>>
>> About the rewrite comment: that did not come from a developer, and is
>> definitely overstated. In any case, rewrites are not inherently bad
>> news, there's a bunch of OpenZFS videos from last yearss summit in
>> which the developers talk about various things being completely
>> rewritten from scratch, some things more than twice. So kinda par for
>> the course, and given enough time things get rewritten anyway. XFS has
>> had substantial changes over its history including numerous on disk
>> format changes even before it found its way onto Linux.
>
>
> Could be, should be, may be... that's fine - but it all says the same
> thing... they
> don't know how much time it is going to take to fix - and who knows what
> their
> priority is to get around to it.  The advantages over what already is
> available
> don't appear to be that compelling, especially when weighed with the risks.

So you are saying that you started using raid56 when it was brand new,
before it had *any* kind of persistent repairing or device replacement
and only now, due to a bug that manifests remarkably less bad than the
normal behavior of everything else (minus ZFS), are you now
complaining? So basically this is, "I want it now!" complaint. Because
all the available information has been saying don't use raid56 for
production, but you did so anyway. This is a subjective change in your
evaluation. It has nothing to do with the state of Btrfs so you really
shouldn't blame it when your requirements have clearly changed.


>
> When all this started I did some searches and found Kent Overstreet's page
> on
> bcachefs:  https://goo.gl/U0UFfN
>
> He had some words about the different filesystems - and had this to say
> about btrfs:
>
> btrfs, which was supposed to be Linux's next generation COW filesystem -
> Linux's answer to zfs. Unfortunately, too much code was written too quickly
> without focusing on getting the core design correct first, and now it has
> too many design mistakes baked into the on disk format and an enormous,
> messy codebase - bigger that xfs. It's taken far too long to stabilize as
> well - poisoning the well for future filesystems because too many people
> were burned on btrfs, repeatedly (e.g. Fedora's tried to switch to btrfs
> multiple times and had to switch at the last minute, and server vendors who
> years ago hoped to one day roll out btrfs are now quietly migrating to xfs
> instead).

I have heard from a couple developers that it was a victim of its own
hype/success and too many feature additions without equivalent effort
on error reporting, debugging, and fault injection tools. I have yet
to hear a Btrfs developer say the core design or on disk format has
anything to do with the problems, but to the contrary. The comment
it's bigger than XFS is kinda funny, seeing as it does more than XFS,
md, and LVM combined, so a proper comparison would be comparing all of
them to Btrfs minus their user space tools (for sure Btrfs tools do
not come anywhere near the metric ass tonne of switches or
documentation in LVM or mdadm).

The Fedora report is simply nonsense. Fedora made no meaningful
attempt to switch to Btrfs once, let alone multiple times. FESCo
considered and approved it, with conditions attached. Josef kept
pushing it off because he thought it wasn't ready. And then Josef
moved on from Red Hat, and wasn't replaced. Characterizing this as
"tried to switch" and "had to switch at the last minute" is at best
hyperbole. It was a change proposal, and it never met the requirements
of either the proposer or FESCo for it to proceed further. No changes
in default happened in the installer that had to be reverted.

Perhaps the secret of fast and stable fs development is a single
developer authored file system.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Tue, Oct 11, 2016 at 8:52 PM, Chris Murphy 
wrote:

>
> About the rewrite comment: that did not come from a developer, and is
> definitely overstated. In any case, rewrites are not inherently bad
> news, there's a bunch of OpenZFS videos from last yearss summit in
> which the developers talk about various things being completely
> rewritten from scratch, some things more than twice. So kinda par for
> the course, and given enough time things get rewritten anyway. XFS has
> had substantial changes over its history including numerous on disk
> format changes even before it found its way onto Linux.


Could be, should be, may be... that's fine - but it all says the same
thing... they
don't know how much time it is going to take to fix - and who knows what
their
priority is to get around to it.  The advantages over what already is
available
don't appear to be that compelling, especially when weighed with the risks.


When all this started I did some searches and found Kent Overstreet's page
on
bcachefs:  https://goo.gl/U0UFfN

He had some words about the different filesystems - and had this to say
about btrfs:

btrfs, which was supposed to be Linux's next generation COW filesystem -
Linux's answer to zfs. Unfortunately, too much code was written too quickly
without focusing on getting the core design correct first, and now it has
too many design mistakes baked into the on disk format and an enormous,
messy codebase - bigger that xfs. It's taken far too long to stabilize as
well - poisoning the well for future filesystems because too many people
were burned on btrfs, repeatedly (e.g. Fedora's tried to switch to btrfs
multiple times and had to switch at the last minute, and server vendors who
years ago hoped to one day roll out btrfs are now quietly migrating to xfs
instead).

Then there is this quote:
https://www.youtube.com/watch?v=79fvDDPaIoY=18m28s
"Software that is
designed/ intended to be reliable should not go through large periods of
instability only to be written off as "prepubescence"."
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Tue, Oct 11, 2016 at 8:39 PM, Chris Murphy 
wrote:

> On Tue, Oct 11, 2016 at 6:33 PM, Gerald B. Cox  wrote:
> > On Tue, Oct 11, 2016 at 4:48 PM, Chris Murphy 
> > wrote:
>
> >
> > That may be, but all the articles I read suggested "be afraid, be very
> > afraid".
> > In addition, https://goo.gl/V2IyFq
> > basically just comes out and says, not to use it.
>
> That's stale and overstated but I'm fine with dire warnings because
> otherwise people use experimental stuff and flip out when it face
> plants. What's needed are people who can tediously gather an autopsy
> report so it can be made better. And that's pretty much what's
> happening.
>
> Well that link is from their wiki.  Then of course there is this thread
which
kind of steps through it all:  https://goo.gl/rIyJ0R
One of my favorite quotes:

"Wow. So it sees the data strip corruption, uses good parity on disk to

fix it, writes the fix to disk, recomputes parity for some reason but
does it wrongly, and then overwrites good parity with bad parity?
That's fucked."



> About the Fedora default, this recently came up on desktop@ so I'll
> just refer to that:
> https://lists.fedoraproject.org/archives/list/desktop@
> lists.fedoraproject.org/message/T6FNLLPJ7LICKSVTTPS4KSIDHOKUUPNC/


Yeah, and there was also this thread:  https://goo.gl/wWHLVA
Like I said, I believe that ship has pretty much sailed...
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Tue, Oct 11, 2016 at 6:33 PM, Gerald B. Cox  wrote:

>
> I knew that it was experimental when I first set it up years ago, but I
> never imagined
> it would still be experimental in 2016.  I just got tired of waiting, and
> the statement
> that it would all most likely have to be rewritten was just the last straw
> for me.  The
> only reason I was using it was because of the ease and flexibility to run
> Raid5/6.  With
> that apparently nowhere now on the horizon, time to cut my loses and move
> on.


About the rewrite comment: that did not come from a developer, and is
definitely overstated. In any case, rewrites are not inherently bad
news, there's a bunch of OpenZFS videos from last yearss summit in
which the developers talk about various things being completely
rewritten from scratch, some things more than twice. So kinda par for
the course, and given enough time things get rewritten anyway. XFS has
had substantial changes over its history including numerous on disk
format changes even before it found its way onto Linux.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Tue, Oct 11, 2016 at 6:33 PM, Gerald B. Cox  wrote:
> On Tue, Oct 11, 2016 at 4:48 PM, Chris Murphy 
> wrote:
>>
>> > As far as BTRFS
>>
>> > is concerned however, I believe that ship has sailed.  I used it for 4
>> > years, but after the recent news regarding RAID
>>
>> The only news about Btrfs RAID I can think of that you're referring to
>> is the raid5 scrub bug that corrupts parity in a very specific
>> situation. It's a bad bug, but it's also really rare.
>
>
> That may be, but all the articles I read suggested "be afraid, be very
> afraid".
> In addition, https://goo.gl/V2IyFq
> basically just comes out and says, not to use it.

That's stale and overstated but I'm fine with dire warnings because
otherwise people use experimental stuff and flip out when it face
plants. What's needed are people who can tediously gather an autopsy
report so it can be made better. And that's pretty much what's
happening.


> I knew that it was experimental when I first set it up years ago, but I
> never imagined
> it would still be experimental in 2016.  I just got tired of waiting, and
> the statement
> that it would all most likely have to be rewritten was just the last straw
> for me.  The
> only reason I was using it was because of the ease and flexibility to run
> Raid5/6.  With
> that apparently nowhere now on the horizon, time to cut my loses and move
> on.

Uhh OK, well it's an entirely new implementation of parity raid,
unlike md or ZFS ZRAID. It was first merged a bit over three years
ago, and the scrub and device replacement code only came last year.
And even still there's no concept of faulty devices - so you don't get
behaviors like devices being ejected from the array on write error or
massive numbers of reads. It's entirely reasonable to want something
more mature. If your use case requires bitrot and silent data
corruption protection and parity raid, the single option is OpenZFS
ZRAID.

>
>>
>> Anyway it's a bad bug. But it's not correct to assign this bug to the
>> other three raid profiles or Btrfs in general - where it has numerous
>> features not all of which have the same maturity.
>
>
> Agreed, but in my mind the last episode throws some serious shade
> on the entire project.  Once upon a time, there was talk about making the
> default in
> Fedora, but now - not so much.  In the meantime XFS is being improved and
> bcachefs is
> being developed.  Tick tock...

About the Fedora default, this recently came up on desktop@ so I'll
just refer to that:
https://lists.fedoraproject.org/archives/list/desk...@lists.fedoraproject.org/message/T6FNLLPJ7LICKSVTTPS4KSIDHOKUUPNC/


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Tue, Oct 11, 2016 at 4:48 PM, Chris Murphy 
wrote:

> > As far as BTRFS
>
> is concerned however, I believe that ship has sailed.  I used it for 4
> > years, but after the recent news regarding RAID
>
> The only news about Btrfs RAID I can think of that you're referring to
> is the raid5 scrub bug that corrupts parity in a very specific
> situation. It's a bad bug, but it's also really rare.
>

That may be, but all the articles I read suggested "be afraid, be very
afraid".
In addition, https://goo.gl/V2IyFq
basically just comes out and says, not to use it.

I knew that it was experimental when I first set it up years ago, but I
never imagined
it would still be experimental in 2016.  I just got tired of waiting, and
the statement
that it would all most likely have to be rewritten was just the last straw
for me.  The
only reason I was using it was because of the ease and flexibility to run
Raid5/6.  With
that apparently nowhere now on the horizon, time to cut my loses and move
on.


> Anyway it's a bad bug. But it's not correct to assign this bug to the
> other three raid profiles or Btrfs in general - where it has numerous
> features not all of which have the same maturity.


Agreed, but in my mind the last episode throws some serious shade
on the entire project.  Once upon a time, there was talk about making the
default in
Fedora, but now - not so much.  In the meantime XFS is being improved and
bcachefs is
being developed.  Tick tock...
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Tue, Oct 11, 2016 at 4:17 PM, Gerald B. Cox  wrote:
>
> On Tue, Oct 11, 2016 at 2:38 PM, Tomasz Kłoczko 
> wrote:
>>
>> You have still 4th option:
>> - create from snapshots separated boot environment (BE)
>> - do upgrade in BE
>> - restart system from new BE
>>
>> To be honest only tis method solves all hazards which are not listed in
>> your 3 points and additionally you will have possibility to create packages
>> without any post install/uninstall scripts which are biggest risk factor of
>> ANY upgrades.
>
>
> Thats fine, but first of all - those aren't my points - they are from the
> link I included regarding Project Tracer.
> My comment was directed at folks who were concerned about running dnf from a
> terminal within a DE - and
> were interested in some type of risk mitigation now.  Your suggestion
> requires a bit more work.

Running an update in the DE on an out of tree snapshot of the file
system is fairly trivial. The harder part is merging the changes that
happen from snapshot time to reboot time, and managing rollbacks.

> As far as BTRFS
> is concerned however, I believe that ship has sailed.  I used it for 4
> years, but after the recent news regarding RAID

The only news about Btrfs RAID I can think of that you're referring to
is the raid5 scrub bug that corrupts parity in a very specific
situation. It's a bad bug, but it's also really rare. First it
requires a data strip to contain corruption. During scrub, the data
block fails checksum, Btrfs does a good reconstruction from parity and
repairs the bad data strip, but then goes on to *sometimes* wrongly
recomputing parity and overwriting the good parity with bad parity. So
in effect, it has silently transposed the corruption from a data strip
to parity strip. Normal operation, you get your data, uncorrupted. If
you lose a drive, and now that bad parity is needed, it results in a
bad reconstruction of data, which results in EIO because the bad data
fails checksum. So to get this form of data loss you need an already
bad data strip, a scrub that hits this particular bug, and then lose a
device. But you don't get corrupt data propagated upward.

It's uncertain how this manifests on raid6, I haven't tested it. My
expectation is the failed csum from reconstruction using p-parity will
result in another attempt using q-parity, and then fixing up the data
and p-parities if the reconstruction passes the data checksum.

Understand that in the identical situation with mdraid and lvm raid, a
scrub check would only report a mismatch, it wouldn't tell you which
was correct. And if you did a scrub repair, it will assume data strips
are valid, and in this case it'd *always* result in the wrong parity
being written over good.

Anyway it's a bad bug. But it's not correct to assign this bug to the
other three raid profiles or Btrfs in general - where it has numerous
features not all of which have the same maturity.

https://btrfs.wiki.kernel.org/index.php/Status

> I switched everything to XFS.

There are many good and valid reasons to use it.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Tue, Oct 11, 2016 at 2:38 PM, Tomasz Kłoczko 
wrote:

> You have still 4th option:
> - create from snapshots separated boot environment (BE)
> - do upgrade in BE
> - restart system from new BE
>
> To be honest only tis method solves all hazards which are not listed in
> your 3 points and additionally you will have possibility to create packages
> without any post install/uninstall scripts which are biggest risk factor of
> ANY upgrades.
>

Thats fine, but first of all - those aren't my points - they are from the
link I included regarding Project Tracer.
My comment was directed at folks who were concerned about running dnf from
a terminal within a DE - and
were interested in some type of risk mitigation now.  Your suggestion
requires a bit more work.  As far as BTRFS
is concerned however, I believe that ship has sailed.  I used it for 4
years, but after the recent news regarding RAID
I switched everything to XFS.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Tomasz Kłoczko]

 On 10 October 2016 at 00:04, Gerald B. Cox  wrote:

> You have 3 options how to resolve it:
>
>1. You ignore it completely and you patiently wait until your system
>crash.
>2. Do Offline Updates
>.
>3. After restart you carefully restart only those applications and
>daemons, which needs to be restarted.
>
> You have still 4th option:
- create from snapshots separated boot environment (BE)
- do upgrade in BE
- restart system from new BE

To be honest only tis method solves all hazards which are not listed in
your 3 points and additionally you will have possibility to create packages
without any post install/uninstall scripts which are biggest risk factor of
ANY upgrades.
Really Linux needs to start learning from what has been done on Solaris
more than decade ago.
What old SySV packages maintainers found that majority of different
customers problems on applying upgrades where generated by fully
customizable post install/uninstall scripts. This was main driving force on
implementing in IPS something which is called package action which is well
tested sequence of operation in some class of installed resources.

BE has very precise properties. It keeps only one kernel instance in each
BE and boot loader must collect on boot stage informations about available
BEs to present them as boot menu. Whatever needs to be customized as
command line boot parameter needs to be part of BE description. Only with
above approach is possible to stop updating boot entries and stop fighting
with many other problems with which all Linux distros are fighting more
than one and half decade.

Again: to solve all upgrade issues it needs to be synchronized few
technologies related to boot manager, package manager and used file
systems. At least using btrfs, grub2 and rpm with removed post
install/uninstall custom scripts is possible to solve finally all issues.
It is possible to prepare packages in some way allowing to use post
install/uninstall in case not using btrfs but it could be exact amin
decision to accept some well known risk coming with choosing old/legacy
approach.

Whatever would be not done if not done using above idea will be still with
non-zero risk of well known and new issues.
Some people must understand that whatever they will be trying to archive
without radical change which Solaris acceped and implemented decade ago it
will be only WASTE OF TIME.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: *http://lnkd.in/FXPWxH *
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Sun, Oct 9, 2016 at 10:01 AM, Gerald B. Cox  wrote:

> For those wanting to know which processes need to be restarted after
> update,
> there is the appropriately named dnf needs-restarting:
> http://dnf-plugins-core.readthedocs.io/en/latest/needs_restarting.html
>
>

I also just found:  Project Tracer: what you should restart after "dnf
upgrade"
You can read about it here, if you're interested:
https://goo.gl/ZFS29E

Preview snippet:
...

Every time you update your system, your currently running applications
should be restarted otherwise you will be still running those old binaries
before the update.

You have 3 options how to resolve it:

   1. You ignore it completely and you patiently wait until your system
   crash.
   2. Do Offline Updates
   .
   3. After restart you carefully restart only those applications and
   daemons, which needs to be restarted.

I really dislike Offline Updates as I see no reason for going offline for
dozens of minutes. And there is only few scripts which helps you when you
want to follow option 3. Most famous is needs-restarting
 from
yum-utils. But the output is very ... ehm ... unfriendly. And it is very
slow.

...
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Sun, Oct 9, 2016 at 7:33 AM, Nico Kadel-Garcia  wrote:

> I must suggest that "nohup dnf whatever > logfile.out &" is your
> friend for exactly this sort of thing. stdin/stdout/stderr are all
> disconnected from the running session and a visible log is generated.
>

Thanks very much for taking the time to post... very simple and to the
point.

Another method I've found for those concerned about using a terminal under
a DE is to
check out dnf-automatic:   https://fedoraproject.org/wiki/AutoUpdates
There is currently an open bug regarding the use of email notifications
when using
dnf-automatic:  https://bugzilla.redhat.com/show_bug.cgi?id=1279773
but other than that, it appears to work fine.

For those wanting to know which processes need to be restarted after
update,
there is the appropriately named dnf needs-restarting:
http://dnf-plugins-core.readthedocs.io/en/latest/needs_restarting.html

As the documentation mentions:

Note that in most cases a process should survive update of its binary and
libraries
it is using without requiring to be restarted for proper operation.
There are however specific cases when this does not apply.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Nico Kadel-Garcia]

On Fri, Oct 7, 2016 at 4:33 PM, Andrew Lutomirski  wrote:
> On Oct 7, 2016 1:29 PM, "Frank Ch. Eigler"  wrote:
>>
>>
>> >> > [...]  I always run dnf manually from the
>> >> > command line, in a VT logged in as root.  And I can run X while doing
>> >> > this and I've never had a dnf update issue.
>>
>> To the extent that the problem is that dnf gets interrupted when its
>> xterm dies, can that be worked around by dnf SIG_IGN'ing SIGHUP /
>> SIGPIPE?  Users could still deliberately interrupt it with SIGINT, but a
>> terminal closure could in theory let it finish unmolested.
>>
>
> Python is weird^Wspecial.  See the bug I opened.

I must suggest that "nohup dnf whatever > logfile.out &" is your
friend for exactly this sort of thing. stdin/stdout/stderr are all
disconnected from the running session and a visible log is generated.
And this is *much, much* easier than trying to repair legacy
architectures of complexity to preserve the status of an associated
terminal wound through X windows and Gnome and "sophisticated feature
filled architecture of the week".

It doesn't fix the underlying mishandling of terminal session loss.
But it can get you and others trapped through the short term.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Andrew Lutomirski]

On Oct 7, 2016 1:29 PM, "Frank Ch. Eigler"  wrote:
>
>
> >> > [...]  I always run dnf manually from the
> >> > command line, in a VT logged in as root.  And I can run X while doing
> >> > this and I've never had a dnf update issue.
>
> To the extent that the problem is that dnf gets interrupted when its
> xterm dies, can that be worked around by dnf SIG_IGN'ing SIGHUP /
> SIGPIPE?  Users could still deliberately interrupt it with SIGINT, but a
> terminal closure could in theory let it finish unmolested.
>

Python is weird^Wspecial.  See the bug I opened.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Frank Ch. Eigler]

>> > [...]  I always run dnf manually from the
>> > command line, in a VT logged in as root.  And I can run X while doing
>> > this and I've never had a dnf update issue.

To the extent that the problem is that dnf gets interrupted when its
xterm dies, can that be worked around by dnf SIG_IGN'ing SIGHUP /
SIGPIPE?  Users could still deliberately interrupt it with SIGINT, but a
terminal closure could in theory let it finish unmolested.

- FChE
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Fri, 2016-10-07 at 21:11 +0200, Roberto Ragusa wrote:
> On 10/04/2016 08:10 PM, stan wrote:
> 
> > I think I can confirm this advice.  I always run dnf manually from the
> > command line, in a VT logged in as root.  And I can run X while doing
> > this and I've never had a dnf update issue.
> 
> 
> The problem with this is that the VT doesn't have a long history,
> so if you can't really see what dnf has done.
> (e.g. I visually search any rpmsave rpmorig rpmnew warning)

Several ways to handle this have already been discussed upthread, and
another thing you can do is look at the 'dnf history info' output for
the transaction; that contains all the same warnings that are shown
real-time.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Roberto Ragusa]

On 10/04/2016 08:10 PM, stan wrote:

> I think I can confirm this advice.  I always run dnf manually from the
> command line, in a VT logged in as root.  And I can run X while doing
> this and I've never had a dnf update issue.

The problem with this is that the VT doesn't have a long history,
so if you can't really see what dnf has done.
(e.g. I visually search any rpmsave rpmorig rpmnew warning)

-- 
   Roberto Ragusamail at robertoragusa.it
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Fenzi]

On Fri, 7 Oct 2016 00:18:21 -0400
Eric Griffith  wrote:

> I'm just thinking out loud here, but, given that rpm-ostree does not
> use grubby, and we do have the Bootloader Spec, and no other distro
> uses grubby, would it be prudent to take a really hard look at
> whether grubby is still a path we want to walk? 

Well, the bootloader spec has a lot of things missing that we
expect/want to support. Like multibooting, BIOS boot chainloading
booting, etc. 

Matthew Garrett started working on proposed changes to the spec: 
https://www.freedesktop.org/wiki/MatthewGarrett/BootLoaderSpec/
but the owners of the spec didn't seem to interested in those cases and
Matthew went on to other work. 

So, I think we would need to get buyin to get our changes into the spec
and get everyone to agree on it before we would want to move to it. 

kevin



pgp5ADC0CGNLR.pgp
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Eric Griffith]

I'm just thinking out loud here, but, given that rpm-ostree does not use 
grubby, and we do have the Bootloader Spec, and no other distro uses grubby, 
would it be prudent to take a really hard look at whether grubby is still a 
path we want to walk? 

If it is, then more work obviously needs to be put into it to get it where we 
want/need it. Personally, I would love to see btrfs subvolume support, so that 
we could have snapshotting like on Suse, though it appears rpm-ostree would 
negate the need for it, correct?

If it's not a path we want to walk anymore, then let's announce an "Intention 
To Deprecate" to give all interested parties (RHEL) plenty of warning, and 
start figuring out what all will break by the removal of grubby. 

Adam, the only other distro that has serious alternate architecture support, 
AFAIK, is Debian. How do they handle grub2 + non-x86? Likewise, the alternate 
architectures that we support, how do we handle their bootloaders? Are they 
grub-based? Ext/Syslinux based? Grub-legacy? 

I agree with Kevin that grub2 is nonintuitive. But the only other serious 
option we have is bootctl, and I am not sure if that's even a possibility for a 
distro like Fedora. I know Arch has it as an install time option, but I don't 
know it's limitations. 

Cheers!
Eric

> On Oct 6, 2016, at 21:20, Chris Murphy  wrote:
> 
>> On Thu, Oct 6, 2016 at 3:17 PM, Kevin Fenzi  wrote:
>> On Thu, 6 Oct 2016 17:05:45 -0400
>> Eric Griffith  wrote:
>> 
>>> Can anyone answer this relatively simple question: "Why grubby?" I've
>>> seen a number of discussions on various topics surrounding the boot
>>> loader that all seem to devolve into "We would love to support that,
>>> but grubby doesn't, so we can't."
>>> 
>>> At what point does the maintenance burden of using grubby outweigh
>>> its own benefits?
>>> 
>>> I don't ask this rhetorically, or because I particularly want to see
>>> grubby gone. I just don't see the benefit that we get from having
>>> grubby when other distros seem to get by just fine without it, or if
>>> they do use it, it doesn't seem to be getting in their way.
>> 
>> Well, I don't know the full history here, but IMHO, the problem is that
>> the way grub2 does config is not very ideal.
> 
> It's not. And OS discovery is fraught with problems also.
> 
> One alternative is Bootloader Spec drop-in scripts. Fedora's GRUB
> carries a patch to support them in the blscfg.mod, but it implements
> it in an early interpretation of the spec. rpm-ostree creates drop in
> scripts per this spec, but instead of depending on blscfg.mod to use
> them directly, rpm-ostree calls a helper to use the scripts as source
> material to generate new grub.cfg or extlinux.conf (both bootloaders
> are supported). rpm-ostree doesn't use grubby at all.
> 
> 
> 
>> Perhaps other distros have figured out better ways to deal with this, I
>> don't know. If someone wanted to go and survey this and report back
>> that information might be of help.
> 
> I don't have a complete or recent evaluation but as of a couple years
> ago, before Gene Czarcinski passed away, we found no other
> distribution using grubby. Distros we ran into use grub-mkconfig as
> recommended by upstream to obliterate the existing grub.cfg and create
> an entirely new one.
> 
> 
> -- 
> Chris Murphy
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Thu, Oct 6, 2016 at 3:17 PM, Kevin Fenzi  wrote:
> On Thu, 6 Oct 2016 17:05:45 -0400
> Eric Griffith  wrote:
>
>> Can anyone answer this relatively simple question: "Why grubby?" I've
>> seen a number of discussions on various topics surrounding the boot
>> loader that all seem to devolve into "We would love to support that,
>> but grubby doesn't, so we can't."
>>
>> At what point does the maintenance burden of using grubby outweigh
>> its own benefits?
>>
>> I don't ask this rhetorically, or because I particularly want to see
>> grubby gone. I just don't see the benefit that we get from having
>> grubby when other distros seem to get by just fine without it, or if
>> they do use it, it doesn't seem to be getting in their way.
>
> Well, I don't know the full history here, but IMHO, the problem is that
> the way grub2 does config is not very ideal.

It's not. And OS discovery is fraught with problems also.

One alternative is Bootloader Spec drop-in scripts. Fedora's GRUB
carries a patch to support them in the blscfg.mod, but it implements
it in an early interpretation of the spec. rpm-ostree creates drop in
scripts per this spec, but instead of depending on blscfg.mod to use
them directly, rpm-ostree calls a helper to use the scripts as source
material to generate new grub.cfg or extlinux.conf (both bootloaders
are supported). rpm-ostree doesn't use grubby at all.



> Perhaps other distros have figured out better ways to deal with this, I
> don't know. If someone wanted to go and survey this and report back
> that information might be of help.

I don't have a complete or recent evaluation but as of a couple years
ago, before Gene Czarcinski passed away, we found no other
distribution using grubby. Distros we ran into use grub-mkconfig as
recommended by upstream to obliterate the existing grub.cfg and create
an entirely new one.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Zbigniew Jędrzejewski-Szmek]

On Wed, Oct 05, 2016 at 01:57:20PM -, Peter Larsen wrote:
> > I never needed to reboot.  I just keep working on my stuff, when I'm 
> > done I turn the laptop off.  Is there any reason to reboot right after 
> > updating?
> 
> That's actually a very common misunderstanding. People think that "yum/dnf 
> update" leaves their system in a new updated stage. But it doesn't 
> (completely). It never has. Only after a reboot are all your patches applied 
> and active. Existing/running processes are rarely if ever reloaded. So when 
> you update libraries, kernels etc. your system will keep running with the old 
> versions of those libraries loaded.  Remember, in Linux a inode is never 
> deleted until the very last process has released it. So any read file handles 
> will keep a file, even one you cannot see with ls, available.
> 
> The only real complete update you can do is one that does a full reboot. We 
> do have a few tricks with DNF which will attempt to let you know what needs 
> restarting. But you'll find that a good part of our updates requires a 
> restart of most if not all your system, in order for the updates to become 
> fully active.

What you write is mostly true, at least for "user" programs.
Various system services actually get restarted, e.g. all those that
are running as systemd units and are have %systemd_postun_with_restart in %post.
For servers that should be a majority of daemons (but not all, dbus-daemon
being a notable exception). Systemd also supports seamless restart of 
socket-activated
services, so it's even possible to restart daemons that are in live use without
anybody noticing.

For users programs we have nothing like that unfortunately.

But yeah, I'm nitpicking here, the general rule is what you say: one
has to restart the machine to make sure everything has bee restarted.

Zbyszek

> This problem often shows itself on long running servers by a system not 
> coming back up/online after a reboot. And nobody understand why - but it's 
> pretty simple. Nobody tested that things worked after an update, and in most 
> cases that requires a reboot. If you kernel, glibc or network control system 
> gets updated, you'll need to reboot to reload them. Or of course take your 
> network offline with everything running on your box (good luck!).
> 
> So it may look like that "it works just fine" but it's a deception. It's 
> still running the older versions of libraries etc for most processes. If you 
> start looking at what's actually running after your update, you'll find that 
> a good part of your updated packages aren't running (yet) - and old versions 
> are still active.  If you're attempting to apply security updates, this is an 
> important distinction and vital to be compliant. For us ordinary users, it's 
> mostly an annoyance as features we wanted, aren't there right after an 
> update, or programs starts failing after an update, as they attempt to 
> dynamicly load in modules and find them "changed", at times not even 
> compatible with the running older version, and things end up "strange". I've 
> seen fonts go nuts, backgrounds disappear, general themes not working etc. 
> after an update on a workstation, simply because the structure got changed - 
> CSSes looking for files that no longer are there, etc. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Fri, 2016-10-07 at 01:25 +0200, Kevin Kofler wrote:
> Adam Williamson wrote:
> > Huh - that's handy, and I did not actually know --nopostun (something
> > new every day, etc.). It does involve including instructions on how to
> > find the package, though, which would inevitably go stale as it moves
> > from u-t to stable. Still, thanks.
> 
> 
> Seeing how this is not the first time a broken %postun script causes 
> unfixable trouble with updating a package, I think RPM really needs a way 
> for a new version of a package to override the old version's %postun. It 
> would be similar to how triggers work, but it should run INSTEAD OF the 
> original %postun, not in addition to it. (As I understand it, postun 
> triggers unfortunately run in addition to normal %postun.)

Yeah, that would be handy for sure. We're kicking around ways to use a
%pre to subvert the effect of the %postun in the bug report right now,
but having a dedicated mechanism would be handy.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Kofler]

Adam Williamson wrote:
> Huh - that's handy, and I did not actually know --nopostun (something
> new every day, etc.). It does involve including instructions on how to
> find the package, though, which would inevitably go stale as it moves
> from u-t to stable. Still, thanks.

Seeing how this is not the first time a broken %postun script causes 
unfixable trouble with updating a package, I think RPM really needs a way 
for a new version of a package to override the old version's %postun. It 
would be similar to how triggers work, but it should run INSTEAD OF the 
original %postun, not in addition to it. (As I understand it, postun 
triggers unfortunately run in addition to normal %postun.)

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Fenzi]

On Thu, 6 Oct 2016 17:05:45 -0400
Eric Griffith  wrote:

> Can anyone answer this relatively simple question: "Why grubby?" I've
> seen a number of discussions on various topics surrounding the boot
> loader that all seem to devolve into "We would love to support that,
> but grubby doesn't, so we can't." 
> 
> At what point does the maintenance burden of using grubby outweigh
> its own benefits?
> 
> I don't ask this rhetorically, or because I particularly want to see
> grubby gone. I just don't see the benefit that we get from having
> grubby when other distros seem to get by just fine without it, or if
> they do use it, it doesn't seem to be getting in their way.

Well, I don't know the full history here, but IMHO, the problem is that
the way grub2 does config is not very ideal. Most applications when you
want to add some new configuration allow you to just do that and leave
everything else you already set the way you wanted it alone. With grub2
it expects to completely regenerate it's config file every time you
want to add a new entry. 

From a practical standpoint this means if you have several entries that
work just fine and add a new one you could end up with none of them
working instead of just the most recent one failing and allowing you to
go back and use one of the previous (working) ones. 

Perhaps other distros have figured out better ways to deal with this, I
don't know. If someone wanted to go and survey this and report back
that information might be of help. 

kevin



pgpBLPG7rZXuF.pgp
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Thu, 2016-10-06 at 17:05 -0400, Eric Griffith wrote:
> Can anyone answer this relatively simple question: "Why grubby?" I've
> seen a number of discussions on various topics surrounding the boot
> loader that all seem to devolve into "We would love to support that,
> but grubby doesn't, so we can't." 
> 
> At what point does the maintenance burden of using grubby outweigh
> its own benefits?
> 
> I don't ask this rhetorically, or because I particularly want to see
> grubby gone. I just don't see the benefit that we get from having
> grubby when other distros seem to get by just fine without it, or if
> they do use it, it doesn't seem to be getting in their way.

The major reason it exists, AIUI, is that it provides a consistent
interface across different bootloaders (we use bootloaders other than
grub on non-Intel architectures). It's also I think mostly consistent
across grub-legacy and grub2. This is a big thing for big RHEL sites
that want to have consistent bootloader management across disparate
arches and RHEL releases (inc. old ones that still use grub-legacy).

The same situation does exist for Fedora, since we *do* have PPC and
s390 and stuff as secondary arches.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Eric Griffith]

Can anyone answer this relatively simple question: "Why grubby?" I've seen a 
number of discussions on various topics surrounding the boot loader that all 
seem to devolve into "We would love to support that, but grubby doesn't, so we 
can't." 

At what point does the maintenance burden of using grubby outweigh its own 
benefits?

I don't ask this rhetorically, or because I particularly want to see grubby 
gone. I just don't see the benefit that we get from having grubby when other 
distros seem to get by just fine without it, or if they do use it, it doesn't 
seem to be getting in their way.

Cheers!
Eric

> On Oct 6, 2016, at 12:12, Chris Murphy  wrote:
> 
> On Thu, Oct 6, 2016 at 7:43 AM, Tomasz Kłoczko  
> wrote:
> 
> 
>> So how problem of consistent upgrade have been solved on Solaris using ZFS
>> and IPS?
>> ZFS has ability to create snapshot of the vol (RO resource) and create on
>> top of the shapshot clone (RW resource).
>> Whole upgrade process consist from few steps:
>> - find volumes which needs to be snapshoted and cloned
>> - create clones
>> - mount clones as separated tree and perform upgrad
> 
> This is what OSTree is effectively doing now.
> 
> /usr is read only. And updates aren't ever applied to the current file
> system tree. A new tree is created and updated, the bootloader
> configuration is changed, and the system is rebooted (at the user's
> leisure) to get the updated system. i.e. it makes it clear your system
> is not running updates until it's rebooted, which it maybe probably
> who knows.
> 
> rpm-ostree integrates RPM support into OSTree.
> 
> 
>>  This part is crucial. If anything wrong will happen during upgrade still
>> working system is not affected. It is possible to observe state of broken
>> upgrade and produce very precise diagnostic data allowing to fix upgrade
>> process on layer of packages. In other words impact of during upgrade on top
>> of still working system is NULL/ZERO!!!
> 
> Yes, it's the same for rpm-ostree, i.e. the basis for the various
> Atomic Host products Fedora creates. And there's going to be a
> Workstation version of it for Fedora 25 (for very early adopters, it
> won't be at getfedora.org).
> 
> 
> 
>> On Linux at the moment is available btrfs which provides possibility of RW
>> snapshots (equivalent of ZFS clones). All what needs to be added to this
>> layer is btrfs volume attribute indicating that volume needs to be cloned
>> during upgrade in case of more complicated scenarios.
> 
> It's actually kinda complicated to handle the changes that occur from
> the time the current tree is snapshot, the update and reboot happens.
> The current root tree can have changes to /var and /etc happening
> during the update and before reboot, so those need to get merged such
> that you can reboot either the old or new tree and have log and
> journal continuity, and so on. rpm-ostree does this.
> 
> 
>> Another small bit which needs to be sorted is related to install procedures
>> implemented in anaconda and post installation procedures in kernel package.
>> What is missing here? anaconda does not allow now to use /boot on btrfs. It
>> forces use ext3/4.
> 
> It's a grubby bug. It doesn't understand Btrfs subvolumes. Any brave
> person who wants to fix grubby? Someone else already tried, so there's
> code available to look at to help grok the logic of what needs to be
> done; but needs to be implemented in a different way to get approved
> by upstream.
> https://bugzilla.redhat.com/show_bug.cgi?id=864198
> 
> 
> -- 
> Chris Murphy
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Andrew Toskin]

Adam Williamson wrote:
> Next time maybe I'll just say 'screw this' and go play golf instead, if this 
> is the thanks I get for trying to help people out.

Others have thanked you, actually, but parts of this email chain have gotten a 
little heated, so I guess it's worth repeating: Thanks for starting this 
discussion. I learned something, and I'm not the only one :)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Thu, 2016-10-06 at 10:18 +0200, Kevin Kofler wrote:
> Adam Williamson wrote:
> > What exactly would you have had me do? Just sit on this and pretend
> > there was no bug and when we saw more and more people come into #fedora
> > and complain about it, tell them the risk was minimal so they should
> > just suck it up and get on with their lives? I mean, seriously, what do
> > you suggest?
> 
> 
> Tell people the actual workaround?
> 
> rpm -Uvh --nodeps --nopostun \
>   
> http://$MIRROR/updates/testing/25/x86_64/s/systemd-udev-231-8.fc25.x86_64.rpm
> 
> --nopostun will disable the broken scriptlet.
> --nodeps will avoid the hard EVR requirement on the rest of systemd.
> Then a normal dnf update should fix your deps.

Huh - that's handy, and I did not actually know --nopostun (something
new every day, etc.). It does involve including instructions on how to
find the package, though, which would inevitably go stale as it moves
from u-t to stable. Still, thanks.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Lars Seipel]

On Wed, Oct 05, 2016 at 03:26:10PM -0700, Japheth Cleaver wrote:
> I don't know what the dnf equivalent is, but isn't that precisely what the
> 'needs-restarting' command provided?

DNF has the tracer plugin for doing exactly that, including printing a
list of commands to restart the affected services, if possible, or
advising a manual restart of specific programs, the login session or the
whole system otherwise, depending on the package set.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Lars Seipel]

On Thu, Oct 06, 2016 at 08:20:34AM -0700, Adam Williamson wrote:
> On Thu, 2016-10-06 at 10:18 +0200, Kevin Kofler wrote:
> > rpm -Uvh --nodeps --nopostun \
> >   
> > http://$MIRROR/updates/testing/25/x86_64/s/systemd-udev-231-8.fc25.x86_64.rpm
> 
> Huh - that's handy, and I did not actually know --nopostun (something
> new every day, etc.). It does involve including instructions on how to
> find the package, though, which would inevitably go stale as it moves
> from u-t to stable. Still, thanks.

Use "dnf download" or yumdownloader to fetch the package?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Thu, Oct 6, 2016 at 7:43 AM, Tomasz Kłoczko  wrote:


> So how problem of consistent upgrade have been solved on Solaris using ZFS
> and IPS?
> ZFS has ability to create snapshot of the vol (RO resource) and create on
> top of the shapshot clone (RW resource).
> Whole upgrade process consist from few steps:
> - find volumes which needs to be snapshoted and cloned
> - create clones
> - mount clones as separated tree and perform upgrad

This is what OSTree is effectively doing now.

/usr is read only. And updates aren't ever applied to the current file
system tree. A new tree is created and updated, the bootloader
configuration is changed, and the system is rebooted (at the user's
leisure) to get the updated system. i.e. it makes it clear your system
is not running updates until it's rebooted, which it maybe probably
who knows.

rpm-ostree integrates RPM support into OSTree.


>   This part is crucial. If anything wrong will happen during upgrade still
> working system is not affected. It is possible to observe state of broken
> upgrade and produce very precise diagnostic data allowing to fix upgrade
> process on layer of packages. In other words impact of during upgrade on top
> of still working system is NULL/ZERO!!!

Yes, it's the same for rpm-ostree, i.e. the basis for the various
Atomic Host products Fedora creates. And there's going to be a
Workstation version of it for Fedora 25 (for very early adopters, it
won't be at getfedora.org).



> On Linux at the moment is available btrfs which provides possibility of RW
> snapshots (equivalent of ZFS clones). All what needs to be added to this
> layer is btrfs volume attribute indicating that volume needs to be cloned
> during upgrade in case of more complicated scenarios.

It's actually kinda complicated to handle the changes that occur from
the time the current tree is snapshot, the update and reboot happens.
The current root tree can have changes to /var and /etc happening
during the update and before reboot, so those need to get merged such
that you can reboot either the old or new tree and have log and
journal continuity, and so on. rpm-ostree does this.


> Another small bit which needs to be sorted is related to install procedures
> implemented in anaconda and post installation procedures in kernel package.
> What is missing here? anaconda does not allow now to use /boot on btrfs. It
> forces use ext3/4.

It's a grubby bug. It doesn't understand Btrfs subvolumes. Any brave
person who wants to fix grubby? Someone else already tried, so there's
code available to look at to help grok the logic of what needs to be
done; but needs to be implemented in a different way to get approved
by upstream.
https://bugzilla.redhat.com/show_bug.cgi?id=864198


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Thu, 2016-10-06 at 08:20 -0700, Adam Williamson wrote:
> On Thu, 2016-10-06 at 10:18 +0200, Kevin Kofler wrote:
> Adam Williamson wrote:
> What exactly would you have had me do? Just sit on this and pretend
> there was no bug and when we saw more and more people come into #fedora
> and complain about it, tell them the risk was minimal so they should
> just suck it up and get on with their lives? I mean, seriously, what do
> you suggest?
> 
> 
> 
> Tell people the actual workaround?
> 
> rpm -Uvh --nodeps --nopostun \
>   
> http://$MIRROR/updates/testing/25/x86_64/s/systemd-udev-231-8.fc25.x86_64.rpm
> 
> --nopostun will disable the broken scriptlet.
> --nodeps will avoid the hard EVR requirement on the rest of systemd.
> Then a normal dnf update should fix your deps.
> 
> Huh - that's handy, and I did not actually know --nopostun (something
> new every day, etc.). It does involve including instructions on how to
> find the package, though, which would inevitably go stale as it moves
> from u-t to stable. Still, thanks.

Sorry for the missed quotes; evolution's composer seems to have found
itself *yet another* failure mode, where it shows me nicely quoted text
but sends something with no quote characters at all...sometimes I
wonder if someone is doing all this because they lost a bet, or
something.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Thu, 2016-10-06 at 10:18 +0200, Kevin Kofler wrote:
Adam Williamson wrote:
What exactly would you have had me do? Just sit on this and pretend
there was no bug and when we saw more and more people come into #fedora
and complain about it, tell them the risk was minimal so they should
just suck it up and get on with their lives? I mean, seriously, what do
you suggest?



Tell people the actual workaround?

rpm -Uvh --nodeps --nopostun \
  http://$MIRROR/updates/testing/25/x86_64/s/systemd-udev-231-8.fc25.x86_64.rpm

--nopostun will disable the broken scriptlet.
--nodeps will avoid the hard EVR requirement on the rest of systemd.
Then a normal dnf update should fix your deps.

Huh - that's handy, and I did not actually know --nopostun (something
new every day, etc.). It does involve including instructions on how to
find the package, though, which would inevitably go stale as it moves
from u-t to stable. Still, thanks.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Tomasz Kłoczko]

Hi every one,

Reading all ideas about solving issues with upgrading systems from working
systems are more or less ideas of ad choc solving some issues or even more
or less reinventing the wheel. IMO all those ideas will not solve anything
and will only increase total level of entropy. After this will be necessary
sooner or later add even more ad choc workarounds and so on ..

I've mention already that some solutions are close to reinventing the wheel.
Why? Because they've been solved long time ago. To be more precise *more
than decade ago*.

I'm working with rpm (RPM Package Manager) more than two decades (try to
execute "rpm -qa --channgelog | grep kloczek" and you can find one on my
earliest activities still present in any RH based distributions. I've been
maintaining for 3 years PLD which at peak time was withs rpm based
distribution with more than 5k src.rpm packages).
Initially rpm was huge step forward because it's been formalizing many
install/upgrade, uninstall, verifications, building, testing problems under
single hood.
Especially many things related to building packages have been solved very
well. So well that even today only some small improvements time too time
needs to be done.

>From the beginning of the rpm (from time when it was 100% implemented in
perl) compare do SySV packages (used on Solaris and BSD*s) and deb (kind of
only improved new skyfold on top of original SySV packaging tools ideas) up
to now problem of consistent upgrade never been solved completely. Why?
Because man assumption about doing upgrade on working system
image/resources is broken by design idea. As long as during upgrade process
will be deleted some files still used by working processes or will be
reopened by those processes always possibility that those processes will be
not able normally used resources or will be trying to use resources from
wrong version is relatively high.
Whatever could be done on packagemanager are to avoid those icebergs is not
enough and will never solve those two fundamental uncertain scenarios.

So why with existing rpm is not possible to solve upgrade dilemmas is
probably more or less obvious now.
So seems like now is yet another iteration of clashes with rmp limitations
only question is how (and by who?) those problems have been already solved?
Answer is very simple: those problems have been solved almost *decade ago
on Solaris* with introduction two crucial technologies like ZFS (Zeta File
System) and IPS (Image Packaging system). These two bits on maintaining
system resources are interacting very closely and they cannot be used
separately (yes .. atm only).

So how *ALL* upgrade problems have been solved on ideas layer?
Very simple: by assumption that system upgrade will never (ever) will be
done on working system resource.
Someone may scratch his head asking "how it is possible to do upgrade if
system resources are not touched?". Answer is that it is not possible to
implement this idea adding some functionalities to package management (PM)
software. Such operation like upgrade needs to be supported by OS and to be
more precise by *FS layer*.

So how problem of consistent upgrade have been solved on Solaris using ZFS
and IPS?
ZFS has ability to create snapshot of the vol (RO resource) and create on
top of the shapshot clone (RW resource).
Whole upgrade process consist from few steps:
- find volumes which needs to be snapshoted and cloned
- create clones
- mount clones as separated tree and perform upgrad
  This part is crucial. If anything wrong will happen during upgrade still
working system is not affected. It is possible to observe state of broken
upgrade and produce very precise diagnostic data allowing to fix upgrade
process on layer of packages. In other words *impact of *during *upgrade*
on top of still working system *is* *NULL/ZERO!!!*
- when upgrade process is finished grub boot loaded configuration is
updated to add new root point from from which updated system system image
needs to be booted.

As I wrote two technologies here (together) are crucial here to solve 100%
upgrade issues: ZFS and IPS. 3rd minor part is bootloaded. Originally on
Solaris 10 was used grub and grub2 on Solaris 11 only simplified whole
workflow.

So what is missing here on Linux to implement those idea? To be hones ..
not to much which is good :)
Only few small bolts and beans are missing :)

On Linux at the moment is available btrfs which provides possibility of RW
snapshots (equivalent of ZFS clones). All what needs to be added to this
layer is btrfs volume attribute indicating that volume needs to be cloned
during upgrade in case of more complicated scenarios.
Why? Because automatic discovery may be not enough in cases like mayr
database upgrade when part of the u[grade may be some format change which
needs to be applied in format for example database files used by some
application. If in boot loaded will be possible to have to boot entries
allowing to boot from original state from before upgrade 

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Wed, Oct 5, 2016 at 8:43 PM, Adam Williamson 
wrote:

>
> OK, from now on when I see bugs causing serious real world consequences
> for multiple real world reporters I'll just shut up and pretend it's
> not happening, shall I? Wouldn't want to be all 'alarmist' about it.


Adam... those questions in my last message were rhetorical.  I wasn't
expecting an answer.

You're still missing my point.

I appreciate the fact that you took positive action in regards to the
systemd/scriptlet bug.  The
Issue I had was when you made the blanket statement in regards to running
DNF.  That was the
overreach.  It was then compounded IMHO when you started the with the
implication that the
GNOME model should be expanded into other DEs.  As you yourself mentioned,
you want to help people
with this bug - but you expanded that into a mini-crusade for "offline
updates".

As Kevin pointed out, the best solution would be to give people the actual
workaround.

I would consider the concept of "offline updates" to be a sea change in the
mind of most Linux users.
I can imagine the press release:

"Ten Years in the making:  Fedora develops innovative methodology for safer
system updates:  REBOOT"

Now of course that can be spun many different ways, but honestly you can
guess how that is going to
play out in the twitter-verse.

As I mentioned several times, I do understand the value for "offline"
updates.  I also mentioned
that in the RFE Bug:  A nice goal would be for DNF to be able to
automatically recognize
which updates require a reboot and segregate them off for later processing
while continuing with those
which are safe to do in an online environment.  Many people are going to
have an issue with reboot for
all updates - and as well they should.  It just isn't necessary.  Which is
why I suggested it be added as an option,
at least until the logic to handle it automatically could be developed.

Regarding your comment about sharing history, etc. between DNF and PK, it
is apparently being
addressed:
https://lists.fedoraproject.org/pipermail/devel/2015-October/216136.html

If people feel strongly that not doing offline updates is a huge risk then
they should
escalate and take it up with FESCo.  I suspect the reason that the DNF team
isn't giving it a high priority
is because they don't view it as an urgent issue/risk.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Adam Williamson writes:


On Wed, 2016-10-05 at 23:54 -0400, Sam Varshavchik wrote:

> * As a general workaround for this type of crashes, we need a
> >   complete-transaction command in DNF – please add your voices to:
> >   https://bugzilla.redhat.com/show_bug.cgi?id=1091702
> >   – and not the sledgehammer approach of doing all updates offline.
>
> CLOSED/WONTFIX since 2014. No comment.

Except there is a comment. This is what Ales wrote when closing the
bug:

"Closign this as WONTFIX with a vote, i.e. once enough people are CCed
in the bug, we will see about adding this."

IOW, he didn't exactly mean WONTFIX. So, if you want that feature
added...CC yourself.


That's fair enough.

But I'm still waiting for a logical explanation why setsid() plus  
sigaction() for selected signal won't be, at least, an interim fix, insofar  
as preventing a failed install due to an X crashing for some reason, due to  
a bad scriptlet, or something.


The only proposed explanation is that you still won't immediately know if  
the transaction is still running or not; without a clear explanation why  
ps(1) will be insufficient to make that determination. And, of course,  
implicit in that argument is that setsid()+sigaction() *will* be sufficient  
and that's just a different problem to solve. But just because the second  
problem needs a different solution doesn't mean that the first problem  
cannot be solved.





pgp50O4wp7qan.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Kofler]

As for the KDE part:

Adam Williamson wrote:
> The 'standard Fedora solution' for KDE is...well I don't know,

The standard updater on the KDE Spin is plasma-pk-updates. The other tools 
can also do updates (or at least they claim to be able to), but the applet 
in the system tray that automatically notifies users of updates (and can 
also be manually refreshed) and performs them is plasma-pk-updates. The 
intention is that people should not be firing up a windowed application to 
do updates manually at all, they should just OK the automatic notification 
in the system tray (i.e., click on the "Install Updates" button).

> because KDE being KDE it ships three different package management GUIs,

There are 3 different package management GUIs not because "KDE is KDE", but 
because we had to work around at least 2 issues with Apper:
1. a Fedora-specific issue: the PackageKit-hif backend (still) does not
   implement the APIs to enumerate comps groups that Apper needs (which is
   the main reason Discover was added, so that people can browse packages –
   Discover uses AppStream instead of PackageKit for that), and
2. an upstream issue: Apper was not ported to KF5 (there is now an
   experimental port), and Plasma 4 (libplasma 1) widgets/plasmoids cannot
   be used in Plasma 5 (libplasma 2), so the standalone applet
   plasma-pk-updates was written (which is likely to stay because it is
   already better than the Apper plasmoid ever was, it shows information
   Apper displays only if you bring up the full UI).

We are looking into replacing Apper with something working better, but right 
now, the most likely candidate would be dnfdragora + plasma-pk-updates 
(dnfdragora itself has no applet, and is unlikely to get one because it is a 
cross-desktop solution using the toolkit abstraction libyui; Mageia, where 
dnfdragora comes from, is also going to implement plasma-pk-updates), and I 
am not sure Discover is going to be kicked out, either (some people really 
believe in AppStream – if it was up to me, Discover would be shown the door 
as soon as dnfdragora is implemented). Unfortunately, since, unlike the 3 
current tools, dnfdragora does not use PackageKit (but dnfdaemon), it will 
not really mean fewer ways to do updates. We are not going to patch out 
updating support from the package management applications.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Florian Weimer]

On 10/05/2016 03:00 AM, Adam Williamson wrote:

On Tue, 2016-10-04 at 20:43 -0400, Sam Varshavchik wrote:

But ordinary regular app updates will happily run on cruise control, without
bringing the system down into single user mode. If Android can do that, I
see no reason why Fedora can't, either. The only time you need to reboot an
Android device is for a kernel-level update.


No, in fact, it's for any *system level* update. Any change to the
underlying system (as opposed to an app) requires the full reboot
treatment. Only updates to app packages don't.


Even that depends on what the “app” is doing.  If the format of data 
files changes in an incompatible way (or just their location on the file 
system



The reason Android can do fairly good app updates is precisely because
it does exactly what Flatpak and Snappy are trying to do for Linux:
hard separation between app space and system space.


I seriously doubt that is the reason.

This looks much more relevant:




Basically, the system can kill applications which do not run in the 
foreground at *any* time to recover resources.  This facility can also 
be used to transparently update system components on which applications 
depend.


> We can't realistically do it with the 'distribution is just a big pile
> of RPMs' model.

The lack of application state management which mirrors what mobile 
platforms are doing is hardly an RPM issue.


Florian

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Florian Weimer]

On 10/06/2016 05:45 AM, Adam Williamson wrote:

On Wed, 2016-10-05 at 20:38 -0700, Andrew Lutomirski wrote:


I don't understand how you arrive at this conclusion. dnf is sitting

on the top of a house of cards when it's running in Terminal. If
anything below it dies, dnf dies and by extension so is rpm. Could dnf
be put into it's own session or scope (whatever it's called), and
would that prevent it from dying if the whole GUI died?



Yes.  dnf would prepare the transaction, solve dependencies, etc, and
then kick off a service to do the actual work.  The service would pipe
the progress state and such back to the client.


FWIW, someone told me today (in rather colorful terms) that apt-get is
designed exactly this way. It survives the death of its controlling
terminal (and, allegedly - again, I don't know this from personal
experience - has decent support for completing failed transactions if
the apt-get process itself crashes).


apt-get and dpkg cannot survive arbitrary crashes, but in general, the 
combination will rerun the equivalent of %preinst and %postinst scripts, 
to make sure that all installed packages are in a properly configured state.


It also intercepts ^C, so that you don't end up with an inconsistent 
package database simply because the administrator hit ^C in the wrong 
terminal.


Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Kofler]

Adam Williamson wrote:
> What exactly would you have had me do? Just sit on this and pretend
> there was no bug and when we saw more and more people come into #fedora
> and complain about it, tell them the risk was minimal so they should
> just suck it up and get on with their lives? I mean, seriously, what do
> you suggest?

Tell people the actual workaround?

rpm -Uvh --nodeps --nopostun \
  http://$MIRROR/updates/testing/25/x86_64/s/systemd-udev-231-8.fc25.x86_64.rpm

--nopostun will disable the broken scriptlet.
--nodeps will avoid the hard EVR requirement on the rest of systemd.
Then a normal dnf update should fix your deps.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Wed, 2016-10-05 at 23:54 -0400, Sam Varshavchik wrote:

> * As a general workaround for this type of crashes, we need a
> >   complete-transaction command in DNF – please add your voices to:
> >   https://bugzilla.redhat.com/show_bug.cgi?id=1091702
> >   – and not the sledgehammer approach of doing all updates offline.
> 
> CLOSED/WONTFIX since 2014. No comment.

Except there is a comment. This is what Ales wrote when closing the
bug:

"Closign this as WONTFIX with a vote, i.e. once enough people are CCed
in the bug, we will see about adding this."

IOW, he didn't exactly mean WONTFIX. So, if you want that feature
added...CC yourself.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Kevin Kofler writes:


Adam Williamson wrote:
> I don't want to get in the KDE folks' bad graces, but this likely could
> also affect KDE's graphical update system, so I'd advise against using
> that for the present too.

Offline updates in KDE are just not going to happen any time soon.


I also don't see offline updates as a reasonable solution to this problem:

* For this particular crash, the bug should simply be tracked down and
  fixed.


Thank you. I'm glad to see that I have not lost my sanity.


* As a general workaround for this type of crashes, we need a
  complete-transaction command in DNF – please add your voices to:
  https://bugzilla.redhat.com/show_bug.cgi?id=1091702
  – and not the sledgehammer approach of doing all updates offline.


CLOSED/WONTFIX since 2014. No comment.




pgpM8kb4wZ9Lx.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Wed, 2016-10-05 at 19:17 -0700, Gerald B. Cox wrote:
> On Wed, Oct 5, 2016 at 3:53 PM, Adam Williamson 
> wrote:
> 
> > I honestly think GNOME has this exactly right for the 'general' user:
> > the safe thing to do is to strongly encourage offline updates, i.e.
> > don't offer any online update mechanism through the desktop. In a
> > completely practical sense, given the current state of the tools and
> > the fact that we know bugs like this crop up - not *often*, but more
> > than *never* - I think it's a more responsible approach than running
> > the update process inside the desktop session.
> > 
> > Could KDE perhaps look into making it so the update process runs
> > outside of the desktop session somehow, if it's not going to go to
> > full-on offline updates 'any time soon'? I know this decision has its
> > own complexities, though.
> > 



> OK, I can see the value in offline updates, so I create a RFE to DNF, which
> I thought was suppose
> to be THE STANDARD FEDORA SOLUTION.

Hmm, well, it's not really that simple, is it? I mean, if you just
landed from Mars and installed Fedora Workstation, you'd have no idea
what dnf was. It's not remotely discoverable. I haven't read the user
guide in forever so I dunno what that says, but no-one reads the docs
anyway. What you'd actually find by poking around is GNOME Software.
Which isn't dnf at all. The 'STANDARD FEDORA SOLUTION' for Workstation
is offline updates with GNOME Software. The 'standard Fedora solution'
for KDE is...well I don't know, because KDE being KDE it ships three
different package management GUIs, but it's probably not really dnf
either. dnf is the STANDARD FEDORA SOLUTION for Server and Cloud, I
guess, but you're quite unlikely to be running it from inside X on
either of those.

> Where was the campaign to communicate this "serious issue" to the Fedora
> community?

There wasn't one. I took it upon myself to send out an email because
I'd seen enough reports of people getting broken updates - there had
been I think seven or eight reports in #fedora at that point, and we
had a live reporter in #fedora-qa who was helping us pin down the bug.
Next time maybe I'll just say 'screw this' and go play golf instead, if
this is the thanks I get for trying to help people out. I've already
been told today that I have 'NO FUCKING CLUE', and you're not helping.
At this point I'm not sure exactly why I bothered.

> Do you think you're going to get the message out by posting to the
> development mailing lists?

The original mail was CC'ed to users@. I wrote a detailed blog post on
this yesterday:

https://www.happyassassin.net/2016/10/04/x-crash-during-fedora-update-when-system-has-hybrid-graphics-and-systemd-udev-is-in-update/

I posted that to the thread (also CC'ed to users@), and I also tweeted
and G+'ed it. Either it or the original email has been picked up by
enough news outlets (usually the original email, because of course news
outlets don't bother to research follow-ups with actual details, sigh)
that at this point my inbox is full of 'haha systemd sucks' flames.
Which of course also makes me feel great about things.

> Where was this discussion when we went from YUM to DNF if it was such an
> issue?

This didn't change at all between yum and dnf. yum wasn't written to
try and survive its console dying, and neither is dnf. I don't know if
changing that was considered.

I mean, again, same question I asked Sam: what exactly is your *goal*
here? What are trying to achieve by hounding this thread? We're not
sending the Update Police around to your house to tell you how to
update your systems. If you've decided that you're okay with the risks
and you want to keep running 'dnf update' from a terminal in X I'm not
going to *stop* you. You can go right ahead. All I know is we were
getting multiple reports of multiple real users getting their systems
into messed up states due to this bug, so it seems like a responsible
idea to alert people to the issue and suggest that they take steps to
avoid it.

If we'd had all the precise details about the cause and the affected
hardware at the time I wrote the original email, I'd have included it.
In fact if I'd known we were going to get that detail quite soon after
I sent the email, I'd have held off and sent a more detailed mail
later. But I didn't know that, and I didn't want to sit on the issue
for an undetermined amount of time, so I wrote about it. I'm terribly
freaking sorry that it seems to have caused you such strife.

> If this is such a huge issue, why doesn't the DNF team consider it a higher
> priority?

I don't know. I'm not the DNF team. Maybe they don't believe many
people run it from inside X. Maybe they believe people run it in tmux
or screen. I'm a QA monkey. I just deal with bugs. All goddamn day
long, I deal with bugs. When there's a sufficiently serious one, I try
to tell people about it.

> Why are we asking that each DE reinvent the wheel on this when we have

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Wed, 2016-10-05 at 20:38 -0700, Andrew Lutomirski wrote:

> I don't understand how you arrive at this conclusion. dnf is sitting
> > on the top of a house of cards when it's running in Terminal. If
> > anything below it dies, dnf dies and by extension so is rpm. Could dnf
> > be put into it's own session or scope (whatever it's called), and
> > would that prevent it from dying if the whole GUI died?
> 
> 
> Yes.  dnf would prepare the transaction, solve dependencies, etc, and
> then kick off a service to do the actual work.  The service would pipe
> the progress state and such back to the client.

FWIW, someone told me today (in rather colorful terms) that apt-get is
designed exactly this way. It survives the death of its controlling
terminal (and, allegedly - again, I don't know this from personal
experience - has decent support for completing failed transactions if
the apt-get process itself crashes).
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Andrew Lutomirski]

On Tue, Oct 4, 2016 at 10:37 AM, Chris Murphy  wrote:
> On Tue, Oct 4, 2016 at 11:26 AM, Andrew Lutomirski  wrote:
>> On Tue, Oct 4, 2016 at 10:05 AM, Kevin Fenzi  wrote:
>>> On Tue, 4 Oct 2016 09:51:16 -0700
>>> Andrew Lutomirski  wrote:
>>>
 By that standard, why do we support dnf at all?

 $ sudo dnf upgrade
 Error: dnf upgrade is dangerous.  Use PackageKit instead and reboot
 when asked.

 I, for one, *like* not rebooting, and I'm perfectly capable of
 rebooting manually if stuff breaks.  As far as I know, Fedora
 considers plain ol' dnf to be supported.
>>>
>>> Well, the problem there, what do you mean by 'support'?
>>>
>>> In this case lots of people use dnf for updates, so IMHO it would be
>>> "we will try and keep this working, and fix anything we can, but do
>>> understand that there's a low level problem here that something could
>>> mess up updates in progress, if you want to be more sure of not hitting
>>> problems, use the offline updates in your graphical desktop"

 For server use, I'm not convinced that the offline update mechanism is
 supported (at the very least, I have no idea how to trigger it), and
 servers have the same issue.
>>>
>>> Much less so. In the server case you have usually ssh, bash and dnf, in
>>> the desktop case you have X, possibly wayland, tons of graphics
>>> libraries, the terminal application you are using and all it's
>>> libraries, and a shell and dnf. There's just a lot more there to
>>> possibly crash.
>>
>> My point is that a lot of this exposure could be avoided.  Sure,
>> there's a decent chance that updating packages will crash running
>> programs.  But, unless one of those programs is dnf, rpm, or systemd,
>> that shouldn't be an excuse to blow up the whole upgrade.
>
>
> I think it's avoided by propagating the point adamw started the thread for.
>
> User: Doctor, it hurts when I do this!
> DocAdamW: So then don't do that!
>
> Do users need an INFO message when running 'dnf update' to kill off
> the myth that without a warning it's expected to be reliable? Maybe.
>

I've pondered this a bit, and I think I disagree with you.
A more apt analogy would be:

User: Doctor, it hurts when I walk.
Doctor: So don't walk.

The System Administrator's Guide tells people to use dnf [1].  I agree
that, in the long run, online updates of the system are probably the
wrong solution.  But the long run isn't here, and we could do a whole
lot better than we do right now with a straightforward change.

>
>
> > I've had
> > Firefox blow up many times due to concurrent dnf, but this doesn't
> > hose my system.  Having gnome-terminal or X or Wayland die shouldn't
> > be any more dangerous.
>
> I don't understand how you arrive at this conclusion. dnf is sitting
> on the top of a house of cards when it's running in Terminal. If
> anything below it dies, dnf dies and by extension so is rpm. Could dnf
> be put into it's own session or scope (whatever it's called), and
> would that prevent it from dying if the whole GUI died?

Yes.  dnf would prepare the transaction, solve dependencies, etc, and
then kick off a service to do the actual work.  The service would pipe
the progress state and such back to the client.

> Even if true,
> how does the user know to wait XX minutes before hitting the reset
> button? I think it's a rabbit hole, just stop touching the owie.
>

In my entire experience administering Linux machines, I have *never*
had serious package problems due to a transaction failing for any
reason *other* than loss of the session that the package manager is
running in.  So I really do think it's a 90% solution to the problem
that doesn't even require changing anyone's workflow.

So I filed the RFE.  I think this should also be a prerequisite for
the KillUserProcesses change.

https://bugzilla.redhat.com/show_bug.cgi?id=1382224

[1] 
https://docs.fedoraproject.org/en-US/Fedora/24/html/System_Administrators_Guide/sec-Installing.html
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Wed, Oct 5, 2016 at 3:53 PM, Adam Williamson 
wrote:

> I honestly think GNOME has this exactly right for the 'general' user:
> the safe thing to do is to strongly encourage offline updates, i.e.
> don't offer any online update mechanism through the desktop. In a
> completely practical sense, given the current state of the tools and
> the fact that we know bugs like this crop up - not *often*, but more
> than *never* - I think it's a more responsible approach than running
> the update process inside the desktop session.
>
> Could KDE perhaps look into making it so the update process runs
> outside of the desktop session somehow, if it's not going to go to
> full-on offline updates 'any time soon'? I know this decision has its
> own complexities, though.
>

OK, I just think we need to do a reality check here.  I initially was
trying to find
out exactly what was the risk... all the posts I read was more or less 99%
of the time
you'll be fine, but "be afraid, be very afraid".  I then asked for more
clarification "oh, it
is too big to clarify, but be afraid, be very afraid... this is a serious
issue".  (Oh but by the
way, most of the people who claim it is a serious issue go ahead and do
online updates anyway...)

OK, I can see the value in offline updates, so I create a RFE to DNF, which
I thought was suppose
to be THE STANDARD FEDORA SOLUTION.  The response, which I completely
understood and
agreed with was basically, yeah, this is a good idea... but it is a low
priority.  The implication is that
they believe there are more important items which demand their attention.

First of all, if this was such a serious issue, Fedora completely failed in
making it known.  From
what I gather the target audience for "offline" updates was novice users.
What follows are rhetorical
questions:

Where was the campaign to communicate this "serious issue" to the Fedora
community?
Do you think you're going to get the message out by posting to the
development mailing lists?
Where was this discussion when we went from YUM to DNF if it was such an
issue?
If this is such a huge issue, why doesn't the DNF team consider it a higher
priority?
Why are we asking that each DE reinvent the wheel on this when we have
DNF?  That just seems
to be a complete waste of resources.

Every software has risks.  I have yet to have an issue using YUM or DNF for
online updates.  The only
time I have experienced an issue (which BTW was a complete PITA) was with
PackageKit.  So much
for that...

If Fedora collectively believes this is a serious enough issue then get the
DNF team to change their
priorities - otherwise people need to consider the risk in association with
the exposure.  Everything that
I have read indicates that it is minimal.  That isn't to say it shouldn't
be done... what I am saying is
people need to stop being alarmist and be worried about more serious
issues.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Thu, 2016-10-06 at 00:05 +0200, Kevin Kofler wrote:
> Adam Williamson wrote:
> > I don't want to get in the KDE folks' bad graces, but this likely could
> > also affect KDE's graphical update system, so I'd advise against using
> > that for the present too.
> 
> 
> Offline updates in KDE are just not going to happen any time soon.
> 
> 
> I also don't see offline updates as a reasonable solution to this problem:
> 
> * For this particular crash, the bug should simply be tracked down and
>   fixed.

We already did that, but two things:

1) that doesn't do a damn thing to help anyone who ran into it before
we tracked it down and fixed it. Offline updates sure were a great
*mitigation* before we fixed the bug.

2) the short-term fix is to remove the service restart from the package
scriptlets, but because the service restart was in the %postun
scriptlet, the update can't fix it immediately: indeed, *installing*
the update will inevitably trigger the bug. The update can only prevent
it happening for *subsequent* updates.

> * As a general workaround for this type of crashes, we need a
>   complete-transaction command in DNF – please add your voices to:
>   https://bugzilla.redhat.com/show_bug.cgi?id=1091702
>   – and not the sledgehammer approach of doing all updates offline.

I'm certainly not opposed to improving the experience of recovering
from this kind of crash, but I'm really not a big fan of the message
'if the update process happens to crash X, hope you understood what
happened, and here's the magic recovery command you can run'.

I honestly think GNOME has this exactly right for the 'general' user:
the safe thing to do is to strongly encourage offline updates, i.e.
don't offer any online update mechanism through the desktop. In a
completely practical sense, given the current state of the tools and
the fact that we know bugs like this crop up - not *often*, but more
than *never* - I think it's a more responsible approach than running
the update process inside the desktop session.

Could KDE perhaps look into making it so the update process runs
outside of the desktop session somehow, if it's not going to go to
full-on offline updates 'any time soon'? I know this decision has its
own complexities, though.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Japheth Cleaver]

On 10/5/2016 2:40 PM, Chris Murphy wrote:

On Wed, Oct 5, 2016 at 3:14 PM, Przemek Klosowski
 wrote:


I think it is a mistake to require reboot for _every_
update, even though, as you say, even user apps sometimes _cannot_ be
updated online (*). I am comfortable with online updates, and I take more
drastic action---restart apps and/or reboot---only when I actually see
problems.

If you read all the posts I cited, it's pretty clear that the notion
updates are really being used if you haven't rebooted is entirely
illusory/wishful thinking. There's a good chance you wouldn't know
there were a problem.The only way to be sure is to reboot.

I don't know what the dnf equivalent is, but isn't that precisely what 
the 'needs-restarting' command provided?


And speaking more broadly, a competent RH-derived-distro administrator 
is already aware of this concern about libraries not taking effect. They 
also have the tools (lsof, etc.) at their disposal to make their own 
decisions on a live system about whether a service (or the host itself, 
for things like glibc) should be bounced when updating a library package.


It feels like any auto-restart behavior (outside of the %post scriptlets 
in the service packages themselves) should be advisory in nature.


-jc
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Kofler]

Adam Williamson wrote:
> I don't want to get in the KDE folks' bad graces, but this likely could
> also affect KDE's graphical update system, so I'd advise against using
> that for the present too.

Offline updates in KDE are just not going to happen any time soon.


I also don't see offline updates as a reasonable solution to this problem:

* For this particular crash, the bug should simply be tracked down and
  fixed.

* As a general workaround for this type of crashes, we need a
  complete-transaction command in DNF – please add your voices to:
  https://bugzilla.redhat.com/show_bug.cgi?id=1091702
  – and not the sledgehammer approach of doing all updates offline.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Przemek Klosowski]

On 10/05/2016 05:24 PM, Matthew Miller wrote:

On Wed, Oct 05, 2016 at 04:59:57PM -0400, Przemek Klosowski wrote:

Now that systemd kills processes on logout, and session timeout is
required by security policies in many workplaces including mine, I
fear we'll have more of those. I certainly had update sessions that
involved hundreds of packages and took longer than the session
timeout, whcih is often on ther order of several minutes.

Please remember that we have deferred enabling the
kill-processes-on-logout setting until at least F26.

How is your session timeout implemented? If it's $TMOUT in bash, that
shouldn't affect the _running_ command. Does it let you leave
long-running processes with screen or tmux?

I believe it's a setting in sshd. I will have to learn to run many 
things in tmux/screen: certainly dnf, but also all the science codes 
that we normally do. Actually, this will build our character: currently 
we kind-of depend on getting lucky if the command happens to run in 
background or knows to put itself in background on receivng a HUP.



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Kofler]

Stephen Gallagher wrote:
> That seems like a waste of effort, considering we have the offline updates
> process which just boots into a special, minimalist environment with
> almost nothing but the updater running.

But the offline process is highly impractical, it requires you to interrupt 
all you were doing on every single update, just for the unlikely event that 
doing otherwise would crash X11. Making it so that the X11 crash will not 
crash the update process is, sure, slightly less effective (doesn't prevent 
potential data loss in the unlikely case X11 crashes), but also much more 
practical, while still providing the same reliability guarantees for the 
update process itself. So I don't see this being a waste of effort at all.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Rafal Luzynski]

5.10.2016 22:21 Dan Book  wrote:
> [...] I'd like to add that at least the MATE and Cinnamon spins, possibly
> others, do not include PackageKit and instead expect users to update using
> yumex-dnf or dnf itself. So ideally an offline update mechanism can be added
> to dnf, and then exposed in yumex-dnf. But we may have to consider installing
> PackageKit in those spins. My concern with this is that PackageKit and dnf do
> not share history and many users are used to using dnf.

What about implementing yumex-pk, I mean another tool forked from
yumex but backed by PackageKit, and eventually dropping dnf and
leaving PK as the only tool, both commandline and GUI?

I know we have GNOME Software which already does it but in
more high level and less power user oriented manner.

Rafal
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Wed, Oct 5, 2016 at 3:14 PM, Przemek Klosowski
 wrote:
> On 10/04/2016 08:12 PM, Chris Murphy wrote:
>
> The first is what happens on the other 90% of the world's computers,
> including Android system updates. Consider that. The second exists,
> right now, and it works and it's a lot better than what the rest of
> the world uses.
>
> What do you mean by 'system'? Android apps update online just fine.

It depends on what's being updated. If Google Play is updated, I've
experienced erratic and sluggish applications until the phone is
rebooted. System is anything that's a package rather than the blobs
they stick into various partitions and mandate a reboot to apply.


> I think it is a mistake to require reboot for _every_
> update, even though, as you say, even user apps sometimes _cannot_ be
> updated online (*). I am comfortable with online updates, and I take more
> drastic action---restart apps and/or reboot---only when I actually see
> problems.

If you read all the posts I cited, it's pretty clear that the notion
updates are really being used if you haven't rebooted is entirely
illusory/wishful thinking. There's a good chance you wouldn't know
there were a problem.The only way to be sure is to reboot.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Przemek Klosowski]

On 10/04/2016 08:12 PM, Chris Murphy wrote:

The first is what happens on the other 90% of the world's computers,
including Android system updates. Consider that. The second exists,
right now, and it works and it's a lot better than what the rest of
the world uses.


What do you mean by 'system'? Android apps update online just fine. 
Fedora does not have this distinction betwen apps and core system: it's 
just a bunch of packages. I think it is a mistake to require reboot for 
_every_ update, even though, as you say, even user apps sometimes 
_cannot_ be updated online (*). I am comfortable with online updates, 
and I take more drastic action---restart apps and/or reboot---only when 
I actually see problems.



(*) because of running processes, or library/API dependencies, or 
changes in communication/bus protocols.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Matthew Miller]

On Wed, Oct 05, 2016 at 05:14:11PM -0400, Przemek Klosowski wrote:
> What do you mean by 'system'? Android apps update online just fine.
> Fedora does not have this distinction betwen apps and core system:
> it's just a bunch of packages. I think it is a mistake to require
> reboot for _every_ update, even though, as you say, even user apps
> sometimes _cannot_ be updated online (*). I am comfortable with
> online updates, and I take more drastic action---restart apps and/or
> reboot---only when I actually see problems.

This is one of the reasons we _might_ want to consider delivering most
user apps as Flatpaks (and server apps as containers).


-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Matthew Miller]

On Wed, Oct 05, 2016 at 04:59:57PM -0400, Przemek Klosowski wrote:
> Now that systemd kills processes on logout, and session timeout is
> required by security policies in many workplaces including mine, I
> fear we'll have more of those. I certainly had update sessions that
> involved hundreds of packages and took longer than the session
> timeout, whcih is often on ther order of several minutes.

Please remember that we have deferred enabling the
kill-processes-on-logout setting until at least F26. 

How is your session timeout implemented? If it's $TMOUT in bash, that
shouldn't affect the _running_ command. Does it let you leave
long-running processes with screen or tmux?

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Przemek Klosowski]

On 10/04/2016 12:06 PM, Andrew Lutomirski wrote:


How hard would it be to make dnf do the rpm transaction inside a 
proper system-level service (transient or otherwise)?  This would 
greatly increase robustness against desktop crashes, ssh connection 
loss, KillUserProcs, and other damaging goofs.


I once hosed a RHEL5 system when an ssh terminal running yum died.  Sigh.

Now that systemd kills processes on logout, and session timeout is 
required by security policies in many workplaces including mine, I fear 
we'll have more of those. I certainly had update sessions that involved 
hundreds of packages and took longer than the session timeout, whcih is 
often on ther order of several minutes.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Wed, 2016-10-05 at 10:43 -0700, Japheth Cleaver wrote:
> 
> After a check-update preview, I can count the number of times "yum -y 
> update &" has ever failed me over the years on one hand.

That still sounds like somewhere between 1 and 5 times too many. =)

The thing is, the case where dnf crashes/dies in the middle of the
update is a *really nasty* failure case. It doesn't have to happen very
often to be a problem.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Dan Book]

On Wed, Oct 5, 2016 at 11:59 AM, Gerald B. Cox  wrote:
>
>
> I created a RFE bug requesting that the upgrade function in DNF be changed
> to incorporate "offline" upgrades as an option.  If it is really
> an issue, DNF should handle it.
> https://bugzilla.redhat.com/show_bug.cgi?id=1382063
>
>
This would be really nice to have. I'd like to add that at least the MATE
and Cinnamon spins, possibly others, do not include PackageKit and instead
expect users to update using yumex-dnf or dnf itself. So ideally an offline
update mechanism can be added to dnf, and then exposed in yumex-dnf. But we
may have to consider installing PackageKit in those spins. My concern with
this is that PackageKit and dnf do not share history and many users are
used to using dnf.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Wed, 2016-10-05 at 06:18 -0700, Gerald B. Cox wrote:
> On Tue, Oct 4, 2016 at 8:30 PM, Adam Williamson 
> wrote:
> 
> > Querying the package database and installing new packages. A system
> > update, on the other hand, is...it's not *configurable*, really. You
> > say 'update my system, please!' and you get an updated system. That's
> > kind of it. I'm just struggling to think which of the 'features and
> > functionality' of dnf Gerald was talking about in the context of doing
> > a system update.
> > 
> Adam, one of my favorite commands (which has kept me out of trouble) is
> "dnf history".  Chris
> posted some excellent links so I'm going to say more there.

Ah, that is a point: I *really really hate* that transaction history is
not shared between dnf and PackageKit, so 'dnf history' doesn't show
transactions that were run by pkcon or GNOME Software. That drives me
clean up the damn wall.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Japheth Cleaver]

On 10/5/2016 9:37 AM, Kevin Fenzi wrote:

On Wed, 5 Oct 2016 10:56:35 -0500
Bruno Wolff III  wrote:


On Tue, Oct 04, 2016 at 20:42:11 -0700,
   Adam Williamson  wrote:

The one thing I absolutely would advise against: don't do an update
over ssh! Unless you use screen or tmux, of course. But just sshing
in and running an update is a great way to potentially hit trouble.

This isn't as bad as you might think. While I mean to use screen, I
often forget and very rarely have problems as restarting sshd doesn't
shut down existing ssh sessions.

Right, thats the thing... 99.9% of the time you are just fine. :)

kevin


After a check-update preview, I can count the number of times "yum -y 
update &" has ever failed me over the years on one hand.


-jc

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Kevin Fenzi]

On Wed, 5 Oct 2016 10:56:35 -0500
Bruno Wolff III  wrote:

> On Tue, Oct 04, 2016 at 20:42:11 -0700,
>   Adam Williamson  wrote:
> >
> >I'd say broadly speaking both, but the most disruptive and
> >potentially catastrophic effect is when the update process itself
> >crashes or is killed. Because of how RPM transactions work, this
> >generally leaves you with RPM convinced you have two copies of a
> >bunch of packages installed, and cleaning that up is kind of
> >tedious. The more processes are running underneath the dnf process,
> >the more likely the dnf process is to get knocked out by something
> >else. (I don't know if dnf could sensibly be changed to mitigate
> >this issue; it's really not my focus. I just want to try and help
> >real users deal with the software as it currently exists.)  
> 
> package-cleanup --cleandupes still works on f25, though I am not sure
> what the proper dnf version of this is. (I needed this about a week
> ago when an issue triggered by a kernel problem I don't fully
> understand caused an update to be terminated.) You can usually get
> the updates to finish and then clean up the duplicates. Sometimes it
> gets trickier.

From 'man yum2dnf': 

   Detailed table for package-cleanup replacement:


┌─┬─┐
│package-cleanup --dupes  │ dnf repoquery 
--duplicated  │

├─┼─┤
│package-cleanup --leaves │ dnf repoquery 
--unneeded│

├─┼─┤
│package-cleanup --orphans│ dnf repoquery 
--extras  │

├─┼─┤
│package-cleanup --oldkernels │ dnf repoquery 
--installonly │

├─┼─┤
│package-cleanup --problems   │ dnf repoquery 
--unsatisfied │

├─┼─┤
│package-cleanup --cleandupes │ dnf remove 
--duplicated │

├─┼─┤
│package-cleanup --oldkernels │ dnf remove 
--oldinstallonly │

└─┴─┘

> >The one thing I absolutely would advise against: don't do an update
> >over ssh! Unless you use screen or tmux, of course. But just sshing
> >in and running an update is a great way to potentially hit trouble.  
> 
> This isn't as bad as you might think. While I mean to use screen, I
> often forget and very rarely have problems as restarting sshd doesn't
> shut down existing ssh sessions.

Right, thats the thing... 99.9% of the time you are just fine. :) 

kevin



pgpsDY50y4Fls.pgp
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Bruno Wolff III]

On Tue, Oct 04, 2016 at 20:42:11 -0700,
 Adam Williamson  wrote:


I'd say broadly speaking both, but the most disruptive and potentially
catastrophic effect is when the update process itself crashes or is
killed. Because of how RPM transactions work, this generally leaves you
with RPM convinced you have two copies of a bunch of packages
installed, and cleaning that up is kind of tedious. The more processes
are running underneath the dnf process, the more likely the dnf process
is to get knocked out by something else. (I don't know if dnf could
sensibly be changed to mitigate this issue; it's really not my focus. I
just want to try and help real users deal with the software as it
currently exists.)


package-cleanup --cleandupes still works on f25, though I am not sure what the 
proper dnf version of this is. (I needed this about a week ago when an 
issue triggered by a kernel problem I don't fully understand caused an 
update to be terminated.) You can usually get the updates to finish and then 
clean up the duplicates. Sometimes it gets trickier.



The one thing I absolutely would advise against: don't do an update
over ssh! Unless you use screen or tmux, of course. But just sshing in
and running an update is a great way to potentially hit trouble.


This isn't as bad as you might think. While I mean to use screen, I often 
forget and very rarely have problems as restarting sshd doesn't shut down 
existing ssh sessions.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Wed, Oct 5, 2016 at 6:51 AM, Gerald B. Cox  wrote:

>
> That said, they basically already do it with the dnf-system-upgrade
> plugin; so why not just expand
> that a bit.  Also, while i completely understand that it is much easier to
> just use a sledgehammer and say "offline upgrades for everything" -
> we both know that isn't required.  Again, there is a place for "offline"
> updates - and I would like to see that option in DNF - but everything
> has it's place.
>

I created a RFE bug requesting that the upgrade function in DNF be changed
to incorporate "offline" upgrades as an option.  If it is really
an issue, DNF should handle it.
https://bugzilla.redhat.com/show_bug.cgi?id=1382063
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Peter Larsen]

> I never needed to reboot.  I just keep working on my stuff, when I'm 
> done I turn the laptop off.  Is there any reason to reboot right after 
> updating?

That's actually a very common misunderstanding. People think that "yum/dnf 
update" leaves their system in a new updated stage. But it doesn't 
(completely). It never has. Only after a reboot are all your patches applied 
and active. Existing/running processes are rarely if ever reloaded. So when you 
update libraries, kernels etc. your system will keep running with the old 
versions of those libraries loaded.  Remember, in Linux a inode is never 
deleted until the very last process has released it. So any read file handles 
will keep a file, even one you cannot see with ls, available.

The only real complete update you can do is one that does a full reboot. We do 
have a few tricks with DNF which will attempt to let you know what needs 
restarting. But you'll find that a good part of our updates requires a restart 
of most if not all your system, in order for the updates to become fully active.

This problem often shows itself on long running servers by a system not coming 
back up/online after a reboot. And nobody understand why - but it's pretty 
simple. Nobody tested that things worked after an update, and in most cases 
that requires a reboot. If you kernel, glibc or network control system gets 
updated, you'll need to reboot to reload them. Or of course take your network 
offline with everything running on your box (good luck!).

So it may look like that "it works just fine" but it's a deception. It's still 
running the older versions of libraries etc for most processes. If you start 
looking at what's actually running after your update, you'll find that a good 
part of your updated packages aren't running (yet) - and old versions are still 
active.  If you're attempting to apply security updates, this is an important 
distinction and vital to be compliant. For us ordinary users, it's mostly an 
annoyance as features we wanted, aren't there right after an update, or 
programs starts failing after an update, as they attempt to dynamicly load in 
modules and find them "changed", at times not even compatible with the running 
older version, and things end up "strange". I've seen fonts go nuts, 
backgrounds disappear, general themes not working etc. after an update on a 
workstation, simply because the structure got changed - CSSes looking for files 
that no longer are there, etc. 

//
 Peter
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Tue, Oct 4, 2016 at 8:31 PM, Chris Murphy 
wrote:

>
> And really the bottom line is, dnf update is fine the vast majority of
> the time, except when it isn't.  Failures are somewhere in between a
> bug and not at all surprising. And people have been working hard on
> solving this for years.


Chris, first of all thanks much for posting the links.  They basically
reinforced my opinion.  I have no issue whatsoever with
"offline" updates.  It is of course a valid approach to a problem.  My
concern was that the blanket implication about the safety
of using DNF within a DE.  Even your comment that it is "fine... until it
isn't" (which can be said about anything) proves the point.

Packagekit... is "safe until it isn't" - refer to:
https://bugzilla.redhat.com/show_bug.cgi?id=1259865
which by the way caused me a bunch of grief because on a lark one day I
decided to try it out... sucks to be me I guess.

If offline updates have a place (and yes I believe they do) then why isn't
that functionality built into DNF now?  I would assume (and
yes, I know what happens when people ASS-U-ME - ;-) ) that it is because
the DNF team doesn't believe the risk/benefit ratio is
high enough to put it in yet and they believe other features/functionality
are more beneficial.

That said, they basically already do it with the dnf-system-upgrade plugin;
so why not just expand
that a bit.  Also, while i completely understand that it is much easier to
just use a sledgehammer and say "offline upgrades for everything" -
we both know that isn't required.  Again, there is a place for "offline"
updates - and I would like to see that option in DNF - but everything
has it's place.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: ... and Fedora 25! - Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Matthew Miller]

On Wed, Oct 05, 2016 at 12:38:15PM +0200, Martin Kolman wrote:
> > >    KillUserProcesses=yes
> > Ouch!  Forgot about that.
> BTW, was there any progress in making screen & tmux aware of this, so
> that screen and tmux sessions are not killed when the user logs out or
> - as in this case - the graphical session crashes ?

Some, but not complete, so change was deferred to F26. See
https://bugzilla.redhat.com/show_bug.cgi?id=1357426

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Matthew Miller]

On Wed, Oct 05, 2016 at 01:13:35AM +0200, Björn Persson wrote:
> I don't get any notifications at all. I just try to remember to run Yum
> periodically, and there I'm not told which updates are security updates.

If you install fedora-motd, it'll update /etc/motd with pending update
info when you log in over ssh. And, while it's not really documented as
such, just running `motdgen` will give you output like:

 09:22:04 up 11 days, 22:13,  1 user,  load average: 0.03, 0.12, 0.22
Updates Information Summary: available
2 Security notice(s)
1 Bugfix notice(s)
4 Enhancement notice(s)

This is a fairly new thing and could use input and further development.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Tue, Oct 4, 2016 at 8:30 PM, Adam Williamson 
wrote:

> Querying the package database and installing new packages. A system
> update, on the other hand, is...it's not *configurable*, really. You
> say 'update my system, please!' and you get an updated system. That's
> kind of it. I'm just struggling to think which of the 'features and
> functionality' of dnf Gerald was talking about in the context of doing
> a system update.
>

Adam, one of my favorite commands (which has kept me out of trouble) is
"dnf history".  Chris
posted some excellent links so I'm going to say more there.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Ms Sanchez]

I never needed to reboot.  I just keep working on my stuff, when I'm 
done I turn the laptop off.  Is there any reason to reboot right after 
updating?



Cheers,

Sylvia


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Ms Sanchez]

Kevin, now you mention Wayland...  Could it be possible to be a Wayland 
related issue?  I have Nvidia but no Wayland and never an issue updating 
from desktop.



Cheers,

Sylvia


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: ... and Fedora 25! - Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Martin Kolman]

On Tue, 2016-10-04 at 18:19 -0500, Ian Pilcher wrote:
> On 10/04/2016 01:03 PM, Tomasz Torcz wrote:
> > 
> > On Tue, Oct 04, 2016 at 12:31:43PM -0500, Ian Pilcher wrote:
> > > 
> > > 
> > > Can you clarify?  In what circumstances would the dnf command
> > > running
> > > within a screen session not survive an X/desktop crash?
> > 
> > 
> >    KillUserProcesses=yes
> > 
> 
> Ouch!  Forgot about that.
> 
BTW, was there any progress in making screen & tmux aware of this, so
that screen and tmux sessions are not killed when the user logs out or
- as in this case - the graphical session crashes ?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Tue, Oct 4, 2016 at 9:31 PM, Chris Murphy  wrote:
> OK this is from just one thread, pretty much exactly two years ago.
> It's a long thread so these are just extractions I think are useful in
> getting a few different data points about the rationalization of
> offline updates, and context for the use case where they're most well
> suited (or not).
>
> And really the bottom line is, dnf update is fine the vast majority of
> the time, except when it isn't.  Failures are somewhere in between a
> bug and not at all surprising. And people have been working hard on
> solving this for years.
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YAWUZXOTCHWGPZ4RKKN22YSB575IEDIJ/
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/NUJIJEZN5RV6E6DH7P2EM35OJUC3NM75/
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/4HHSIHSQM7HSQRT3KPLLF5MC7FVVTXAJ/
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/36PTCP2G2I4FKZQSYNT4YLR22557ARBA/
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ZARJIYPOJKUACXFYWFHMSU4WHPDB4IPK/
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/6GIGWVDIKHGBEGWS7CWDRI5OSCS7NLYF/
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ODNPCPTWNVFRGBGC2IQUA5VSSKGESAMV/

One more for that thread is good as well...

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/GH5WJZGD3O55IBGAM7IOUB4ZAHCHVUSI/






-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Wed, 2016-10-05 at 13:34 +1000, Jeff Fearn wrote:
> On 5/10/2016 12:36, Sam Varshavchik wrote:
> > Adam Williamson writes:
> > 
> > > All dnf's 'nice features' aren't really there for a system update, are
> > > they?
> > 
> > 
> > Well what are they there for, then?
> > 
> > >   You can use all of its nice features for doing other things.
> > 
> > 
> > Like what? I thought that the only thing dnf does is update the system.
> > Oh, yeah, sure you can query metadata on available packages too.
> 
> 
> I think there is a misunderstanding here on the use of "system".
> 
> I believe Adam is saying that upgrading packages for the underlying OS
> is where the problem is, but you can upgrade packages in other layers of
> the OS fine.
> 
> e.g. upgrading glibc can be an issue, upgrading vi should be fine.
> 
> Upgrading emacs is undefined.

No, that's not what I meant. I was replying to Gerald's post (though I
missed quoting the relevant bit, which led to some confusion):

"The DNF team has done an excellent job in obfuscating the
horror IMO because (knock on wood) I've always been very impressed
with the features and functionality.  Seems a shame that all those nice
bells and whistles will now be hidden behind some gui or pkcon."

I was just confused as to what 'features and functionality' would be
'hidden' by a GUI or pkcon in the case of doing a system update, since
I can't really think what 'features and functionality' you really use
when running dnf update.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Jeff Fearn]

On 5/10/2016 12:36, Sam Varshavchik wrote:
> Adam Williamson writes:
> 
>> All dnf's 'nice features' aren't really there for a system update, are
>> they?
> 
> Well what are they there for, then?
> 
>>   You can use all of its nice features for doing other things.
> 
> Like what? I thought that the only thing dnf does is update the system.
> Oh, yeah, sure you can query metadata on available packages too.

I think there is a misunderstanding here on the use of "system".

I believe Adam is saying that upgrading packages for the underlying OS
is where the problem is, but you can upgrade packages in other layers of
the OS fine.

e.g. upgrading glibc can be an issue, upgrading vi should be fine.

Upgrading emacs is undefined.

Cheers, Jeff.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Wed, 2016-10-05 at 02:59 +, Andrew Toskin wrote:
> This is the first I've heard of any recommendation like this. If
> running `dnf upgrade` from a graphical console is such a big and
> well-known risk, then why isn't it mentioned in the dnf
> documentation? I've posted about this on the dnf Bugzilla.
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1381785
> 
> I'm having a hard time finding anything about this in the Fedora Wiki
> either. If you could recommend any particular reading that could
> explain this some more, I'd appreciate it.

Hmm, well, there isn't necessarily an awful lot of explicit
documentation of this, I guess. I dunno what's in the user guides these
days. When I said we don't recommend updating from a desktop console, I
guess I was thinking mostly of mailing list discussion, IRC, forums
etc. rather than formal documentation. I don't think there really *is*
any How To Update Your System guide, like we have an Upgrading guide.

> I tried to read every message in this email thread, but I'm still not
> clear: It seems the bug that inspired the original post is based on
> certain graphics hardware, but you still say it's best not to run
> system updates from a graphical session at all anyway. Is most of the
> risk related specifically to X and the large software stack that runs
> on it, or is it simply a problem of numbers, where more running
> processes means more things could crash while dnf installs updates? 

I'd say broadly speaking both, but the most disruptive and potentially
catastrophic effect is when the update process itself crashes or is
killed. Because of how RPM transactions work, this generally leaves you
with RPM convinced you have two copies of a bunch of packages
installed, and cleaning that up is kind of tedious. The more processes
are running underneath the dnf process, the more likely the dnf process
is to get knocked out by something else. (I don't know if dnf could
sensibly be changed to mitigate this issue; it's really not my focus. I
just want to try and help real users deal with the software as it
currently exists.)

The effect where the update causes running processes to misbehave is
usually less of a big deal and more just a case of 'eh, restart it or
reboot, no big deal'.

Note I didn't write about using tmux or screen for two reasons:

1) I'm not sure exactly how safe that is from the systemd
KillUserProcesses change
2) I didn't want to talk about somewhat-advanced topics, I wanted to
keep it simple

but if you do actually know what you're doing with tmux/screen, and
someone can clear up the KillUserProcesses thing (or you just make sure
the server process is launched outside of the user session, I guess),
then I think running an update from a tmux/screen session running on a
terminal in a desktop should be nearly as safe as doing it in a VT.

> Fedora Workstation users are apparently recommended to use GNOME
> Software's reboot/update feature; what's the recommended way to
> update all packages on instances of Fedora Server or Fedora Cloud?

I'm not sure we actually *have* an official recommendation. My take
would be that it depends on your risk tolerance. Obviously on a typical
Server / Cloud install you're not going to be running the update in a
graphical desktop session, so that avoids a lot of the risk right
there. It's still technically a bit safer to use the pkcon-driven
offline update process than just to update in a VT or with dnf-
automatic or a cron job or something, but it's a much smaller
difference, I'd say.

The one thing I absolutely would advise against: don't do an update
over ssh! Unless you use screen or tmux, of course. But just sshing in
and running an update is a great way to potentially hit trouble.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Tue, 2016-10-04 at 22:36 -0400, Sam Varshavchik wrote:
> Adam Williamson writes:
> 
> > 
> > All dnf's 'nice features' aren't really there for a system update,
> > are
> > they?
> 
> Well what are they there for, then?

Querying the package database and installing new packages. A system
update, on the other hand, is...it's not *configurable*, really. You
say 'update my system, please!' and you get an updated system. That's
kind of it. I'm just struggling to think which of the 'features and
functionality' of dnf Gerald was talking about in the context of doing
a system update. I think the most I'd ever use would be --enablerepo /
--disablerepo , and --nogpgcheck (and hopefully we'll never need that
one again anyway, now).

But I was talking to Gerald, not you. I'm not really sure why you keep
jumping in and prolonging this thread - you apparently didn't hit the
bug, and you certainly know enough about the tech to drive your own
personal updates however you choose. What exactly is your goal here?

> Hmmm… Right now, running "pkcon get-updates" claims "There are no
> updates  
> available at this time.", meanwhile "dnf upgrade" shows 68 packages
> that can  
> be updated. "pkcon refresh" doesn't change anything.

I don't actually have much experience of driving it through pkcon, I've
only ever used the GNOME Software route, where the refresh button more
or less seems to do what it claims. I don't use the offline updates
stuff on my own servers, I'm okay with just letting dnf-automatic do
them. My personal fuzzy evaluation is that the you get about 90% of the
safety gain going from 'run in a terminal in X' to 'run in a VT',
offline updates get you another 10%, and I'm okay with that tradeoff
for my machines because they're just not terribly important. If I was
running a gajillion dollar server farm I'd probably have different
considerations.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

OK this is from just one thread, pretty much exactly two years ago.
It's a long thread so these are just extractions I think are useful in
getting a few different data points about the rationalization of
offline updates, and context for the use case where they're most well
suited (or not).

And really the bottom line is, dnf update is fine the vast majority of
the time, except when it isn't.  Failures are somewhere in between a
bug and not at all surprising. And people have been working hard on
solving this for years.

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YAWUZXOTCHWGPZ4RKKN22YSB575IEDIJ/
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/NUJIJEZN5RV6E6DH7P2EM35OJUC3NM75/
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/4HHSIHSQM7HSQRT3KPLLF5MC7FVVTXAJ/
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/36PTCP2G2I4FKZQSYNT4YLR22557ARBA/
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ZARJIYPOJKUACXFYWFHMSU4WHPDB4IPK/
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/6GIGWVDIKHGBEGWS7CWDRI5OSCS7NLYF/
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ODNPCPTWNVFRGBGC2IQUA5VSSKGESAMV/


There are a bunch of other threads scattered about on multiple lists
about and why OSTree. About and why LVM thinp snapshots, Btrfs, and
Snapper. Why Chrome OS, Android, and also even CoreOS went with the
A/B partitioning layout they did.



-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Andrew Toskin]

This is the first I've heard of any recommendation like this. If running `dnf 
upgrade` from a graphical console is such a big and well-known risk, then why 
isn't it mentioned in the dnf documentation? I've posted about this on the dnf 
Bugzilla.

https://bugzilla.redhat.com/show_bug.cgi?id=1381785

I'm having a hard time finding anything about this in the Fedora Wiki either. 
If you could recommend any particular reading that could explain this some 
more, I'd appreciate it.

I tried to read every message in this email thread, but I'm still not clear: It 
seems the bug that inspired the original post is based on certain graphics 
hardware, but you still say it's best not to run system updates from a 
graphical session at all anyway. Is most of the risk related specifically to X 
and the large software stack that runs on it, or is it simply a problem of 
numbers, where more running processes means more things could crash while dnf 
installs updates? Fedora Workstation users are apparently recommended to use 
GNOME Software's reboot/update feature; what's the recommended way to update 
all packages on instances of Fedora Server or Fedora Cloud?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Adam Williamson writes:


All dnf's 'nice features' aren't really there for a system update, are
they?


Well what are they there for, then?


  You can use all of its nice features for doing other things.


Like what? I thought that the only thing dnf does is update the system. Oh,  
yeah, sure you can query metadata on available packages too.



   When
you update the system, all you want is...an updated system. pkcon and
GNOME's offline update flow achieve that just fine.


Hmmm… Right now, running "pkcon get-updates" claims "There are no updates  
available at this time.", meanwhile "dnf upgrade" shows 68 packages that can  
be updated. "pkcon refresh" doesn't change anything.


pgpE9gc2bqiHl.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Tue, 2016-10-04 at 18:59 -0700, Gerald B. Cox wrote:
> 
> I've been scrambling reading threads trying to understand what exactly is
> the exposure here.  The only thing I could find that quantified the risk
> was in this kde thread:
> 
> https://goo.gl/m87COz

You're never really going to be able to 'quantify the risk', because we
don't have solid enough data. We don't know exactly how many millions
of people are running Fedora in how many millions of configurations and
just how many of them have ever had a dnf update failure. We don't even
know any single one of those things. You're kind of on a hiding to
nothing there. Point is, live update processes can and do go wrong. We
certainly have enough records of that. Dig through bugzilla and you'll
find plenty. I can think of ~4 cases that actually *did* get reported
and precisely identified since F21. The more stuff running under the
update process, the more likely problems are to happen.

All dnf's 'nice features' aren't really there for a system update, are
they? You can use all of its nice features for doing other things. When
you update the system, all you want is...an updated system. pkcon and
GNOME's offline update flow achieve that just fine.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Gerald B. Cox]

On Tue, Oct 4, 2016 at 5:43 PM, Sam Varshavchik 
wrote:

> Strictly speaking Fedora doesn't make you do the first one, but it's
>> *well* understood for a long time how fragile this is which is why
>> offline updates was created.
>>
>
> Well, this is a surprise to me. I guess my faith in dnf was misplaced.


Here also.  The DNF team has done an excellent job in obfuscating the
horror IMO because (knock on wood) I've always been very impressed
with the features and functionality.  Seems a shame that all those nice
bells and whistles will now be hidden behind some gui or pkcon.

I've been scrambling reading threads trying to understand what exactly is
the exposure here.  The only thing I could find that quantified the risk
was in this kde thread:

https://goo.gl/m87COz

*"...Updating online works 99.8% of the time. The 0.1%
time it will corrupt random bits of your file-system, and 0.1% of the
time it will leave you vulnerable to the security issue you thought
you just "fixed". The only way to fix this so that online updates are
safe is to redesign the centralised shared package model we use for
distributing applications. The workaround is to use offline updates..."*
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Tue, 2016-10-04 at 20:43 -0400, Sam Varshavchik wrote:
> But ordinary regular app updates will happily run on cruise control, without  
> bringing the system down into single user mode. If Android can do that, I  
> see no reason why Fedora can't, either. The only time you need to reboot an  
> Android device is for a kernel-level update.

No, in fact, it's for any *system level* update. Any change to the
underlying system (as opposed to an app) requires the full reboot
treatment. Only updates to app packages don't.

The reason Android can do fairly good app updates is precisely because
it does exactly what Flatpak and Snappy are trying to do for Linux:
hard separation between app space and system space. Flatpak and Snappy
didn't just spring fully formed from a vacuum, they're very obviously
the product of someone using Android and/or iOS and going 'huh, maybe
we should do that'.

We can't realistically do it with the 'distribution is just a big pile
of RPMs' model. GNOME folks thought we could, for a while, then
realized they were wrong.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Chris Murphy writes:


I suggested earlier in the thread that it automatically put itself in
its own scope. But I don't know if that solves this problem, and even
if it does it's only one part of a much bigger set of problems that
can cause updates to implode. In this case, sure maybe dnf survives,
but X is gone, the user has no idea what just happened, they have no
idea dnf is still applying their updates in the background, so they
don't know they shouldn't reboot.


Well, perhaps I misunderstood the situation at hand. But I'm under the  
impression that, right now, dnf just blows up and leaves things in an  
inconsistent state.


Maybe this is too much of a radical idea, but it makes sense to me to  
address this first, and then maybe worry about what is the best way for dnf  
to forge ahead when it realizes that its terminal is gone.


Right now, if I'm in a middle of a dnf update, and I try to start another  
dnf operation, the second operation will fail. So dnf is smart enough to  
exclusive-lock itself. I also have a dim recollection of some plugin that  
said something about blocking system shutdowns, in some context. I'm a bit  
hazy on that, but in any case, this is not something that must be solved on  
day 1. A non-sophisticated user is not going to drop to a terminal prompt,  
and execute dnf manually. I think that it's a reasonable proposition that  
someone who does do that, and ends up killing X for some reason, will be  
able to figure out if dnf is still running, or not, in the background.



When you start making the list of things that have to work exactly
right for system updates in a GUI , you get basically two things:

1. fuck it, we're making the user log out and do the update offline, and;

2. fuck it, we're doing something completely different ala OSTree, ala
Snapper, so the user can do a rollback if things go wrong.

The first is what happens on the other 90% of the world's computers,
including Android system updates.


But ordinary regular app updates will happily run on cruise control, without  
bringing the system down into single user mode. If Android can do that, I  
see no reason why Fedora can't, either. The only time you need to reboot an  
Android device is for a kernel-level update.



Strictly speaking Fedora doesn't make you do the first one, but it's
*well* understood for a long time how fragile this is which is why
offline updates was created.


Well, this is a surprise to me. I guess my faith in dnf was misplaced.

I spent my time in the trenches. I spent a lot of time writing system  
daemons that I expected to recover from SIGKILL automatically. It wasn't  
exactly easy, but it was not an impossible task either. I'm not expecting  
any medals for that, just mentioning the fact that it's not, and should not  
be, considered to be a lost art. As I'm told it is now, apparently.



>> It's why openSUSE has spent a ton of resources, and a few bloody
>> noses, getting completely atomic updates working with Btrfs and
>> snapper, with very fine rollback capabilities.
>
>
> You do not need atomic updates to install a signal handler for SIGHUP or
> SIGTERM. And maybe issue a setsid() call, beforehand.
>
> This shouldn't be rocket science.

OK you clearly don't understand the complexity. So go ahead and do


Yes, I do not. But I am more than willing to be educated. Please explain to  
me the complexity in `setsid()`, and a few calls to `sigaction()`, and maybe  
an occasional `fork()`, here and there. Granted, when bleep hits the fan,  
you wouldn't immediately know where things stand without further digging.  
But, at least you are not going to end up with an unbootable brick, as long  
as you don't panic, and take things one step at a time.



things your way, and when you don't like the result from hitting the
Hurt Me Button, complain and criticize people all you want, wait for
the silence from those who could not give two shits, which means: stop
doing things wrong, start doing them correctly, or fix it all yourself
Mr. Genius. And then go write your own updater.


You know, actually I did, about ten years ago, I think – don't recall the  
exact timeframe. Nothing earth-shattering; just the same basic functionality  
as rpm: basic install/update/delete, and some dependency tracking.


And, by the way, an atomic package commit. If the process was SIGKILL-ed in  
a middle of a transaction – which might've involved a multi-package  
update – the install would finish automatically, when it got restarted. I  
spent some time on that because I recall that at the time rpm would do that  
too. I am certain that at least some point when you started rpm after  
something blew up, it would yell at you that a previous transaction hasn't  
been finished, and offered to clean it up for you.


So, it seems that this doesn't happen anymore. I guess that's the price of  
progress.




pgpeF8ynEen7U.pgp
Description: PGP signature
___
devel 

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Stephen John Smoogen]

OK Everyone.. enough. If you want to keep at it, take it off-list.


-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Chris Adams writes:


Once upon a time, Sam Varshavchik  said:
> I can make any process survive an X shutdown, using an amazing tool
> called "tmux".
>
> Why can't dnf do the same?

Because dnf would have to reimplement tmux's (or screen's) TTY handling,
which is far outside the scope of a package manager.  If you want to run
dnf under tmux/screen, then run it under tmux/screen.


By my rough estimate, the relevant bits of "TTY handling" that are needed to  
survive the terminal going away comprise no more than four or five system  
calls.



> You do not need atomic updates to install a signal handler for
> SIGHUP or SIGTERM. And maybe issue a setsid() call, beforehand.

That's not enough.  dnf prints running information to standard
out/error, which are connected to the terminal via a TTY.  As soon as
the terminal process is killed, the TTY is closed, and the next write by
dnf will cause an error.


Mea culpa. I forgot about SIGPIPE, my fault. Ok, let's just add SIGPIPE to  
the list of blocked signals.


I'm up to six system calls, now.


Now, the error could be ignored, but it is hard to say there's a
legitimate path to follow after such an error.  A system-level program
like dnf should generally not ignore such errors.  Any RPM script that
writes to standard out/error (they're not supposed to do that, but some
do, sometimes inadvertently) will also fail.


Splendid. Have the scripts' stdout redirected to a pipe, that dnf will read,  
and attempt to log it to the terminal on its own, and deal with the fallout.



If you want dnf to survive the terminal exiting, run it under something
that handles that, like tmux or screen.  Heck, alias dnf to "screen dnf"
(or the tmux equivalent) if you like.  Don't try to make dnf reimplement
all that functionality.


Okee-dokee. That's a reasonable proposition. But only if "reasonable" is  
defined in a fairly limited way; namely "work-around for lack of resilience  
and fault tolerance in a criticial system update tool".




pgpJh00Cv_EJx.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Tue, Oct 4, 2016 at 5:49 PM, Sam Varshavchik  wrote:
> Chris Murphy writes:
>
>> > Seems to me it
>> > would be more worthwhile to build in better error recovery within DNF
>> > than
>> > to always require "offline" - especially
>> > since the incidence of failure (at least anecdotally) just isn't that
>> > high.
>>
>> Sufficiently impractical that it's not possible.
>
>
> Wrong.
>
> I can make any process survive an X shutdown, using an amazing tool called
> "tmux".
>
> Why can't dnf do the same?

I suggested earlier in the thread that it automatically put itself in
its own scope. But I don't know if that solves this problem, and even
if it does it's only one part of a much bigger set of problems that
can cause updates to implode. In this case, sure maybe dnf survives,
but X is gone, the user has no idea what just happened, they have no
idea dnf is still applying their updates in the background, so they
don't know they shouldn't reboot.

When you start making the list of things that have to work exactly
right for system updates in a GUI , you get basically two things:

1. fuck it, we're making the user log out and do the update offline, and;

2. fuck it, we're doing something completely different ala OSTree, ala
Snapper, so the user can do a rollback if things go wrong.

The first is what happens on the other 90% of the world's computers,
including Android system updates. Consider that. The second exists,
right now, and it works and it's a lot better than what the rest of
the world uses.

Strictly speaking Fedora doesn't make you do the first one, but it's
*well* understood for a long time how fragile this is which is why
offline updates was created.


>> It's why openSUSE has spent a ton of resources, and a few bloody
>> noses, getting completely atomic updates working with Btrfs and
>> snapper, with very fine rollback capabilities.
>
>
> You do not need atomic updates to install a signal handler for SIGHUP or
> SIGTERM. And maybe issue a setsid() call, beforehand.
>
> This shouldn't be rocket science.

OK you clearly don't understand the complexity. So go ahead and do
things your way, and when you don't like the result from hitting the
Hurt Me Button, complain and criticize people all you want, wait for
the silence from those who could not give two shits, which means: stop
doing things wrong, start doing them correctly, or fix it all yourself
Mr. Genius. And then go write your own updater.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Adams]

Once upon a time, Sam Varshavchik  said:
> I can make any process survive an X shutdown, using an amazing tool
> called "tmux".
> 
> Why can't dnf do the same?

Because dnf would have to reimplement tmux's (or screen's) TTY handling,
which is far outside the scope of a package manager.  If you want to run
dnf under tmux/screen, then run it under tmux/screen.

> You do not need atomic updates to install a signal handler for
> SIGHUP or SIGTERM. And maybe issue a setsid() call, beforehand.

That's not enough.  dnf prints running information to standard
out/error, which are connected to the terminal via a TTY.  As soon as
the terminal process is killed, the TTY is closed, and the next write by
dnf will cause an error.

Now, the error could be ignored, but it is hard to say there's a
legitimate path to follow after such an error.  A system-level program
like dnf should generally not ignore such errors.  Any RPM script that
writes to standard out/error (they're not supposed to do that, but some
do, sometimes inadvertently) will also fail.

If you want dnf to survive the terminal exiting, run it under something
that handles that, like tmux or screen.  Heck, alias dnf to "screen dnf"
(or the tmux equivalent) if you like.  Don't try to make dnf reimplement
all that functionality.

-- 
Chris Adams 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Adam Williamson]

On Tue, 2016-10-04 at 19:47 -0400, Sam Varshavchik wrote:
> 
> I'm quite astonished to read this. Something is wrong, here. Either I have  
> suffered a stroke, today, or I seriously misunderstood some POSIX  
> fundamentals, for the last twenty years.

Oh fer Pete's sake, would you please ratchet down the rhetoric and stop
posting the same thing over and over again? You've made your position
clear and now you're just polluting the thread. I'm trying to advise
real users on how to ensure they don't suffer real problems, not start
an epic argument about how to write update tools. I'm sure the dnf
developers are happy to take submissions through the usual channels.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Chris Murphy writes:


> Seems to me it
> would be more worthwhile to build in better error recovery within DNF than
> to always require "offline" - especially
> since the incidence of failure (at least anecdotally) just isn't that high.

Sufficiently impractical that it's not possible.


Wrong.

I can make any process survive an X shutdown, using an amazing tool called  
"tmux".


Why can't dnf do the same?


It's why openSUSE has spent a ton of resources, and a few bloody
noses, getting completely atomic updates working with Btrfs and
snapper, with very fine rollback capabilities.


You do not need atomic updates to install a signal handler for SIGHUP or  
SIGTERM. And maybe issue a setsid() call, beforehand.


This shouldn't be rocket science.



pgp59ZDjiz3rX.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Adam Williamson writes:


It's pretty simple, really: a process running in a terminal inside a
graphical desktop will crash if the terminal app crashes, or if the
desktop crashes, or if X crashes.


It should take me about five minutes to write a process that will continue  
happily along if its terminal, desktop, or X crashes. Especially if the  
process doesn't need to talk to X in the first place. Like dnf.


I'm quite astonished to read this. Something is wrong, here. Either I have  
suffered a stroke, today, or I seriously misunderstood some POSIX  
fundamentals, for the last twenty years.


For chrissakes, there's a bleeping program in Fedora called "tmux" that has  
managed to accomplished the herculean feat of surviving X shutting down.


I just performed a simple experiment.

1. Opened a terminal window

2. Executed tmux

3. In a tmux session executed "cat >/dev/null"

4. Logged out from the desktop, and only because CTRL-ALT-BACKSPACE has been  
disabled "for our convenience", but to the tmux sesssion running in a  
terminal this is indistinguishable from losing the X session unexpectedly.


5. Guess what? After logging back in, opening a terminal window, and  
executed "tmux attach", I found my "cat" program happily waiting to dump  
more stuff into /dev/null.


Really, the silly excuse that, somehow, X shutting down is a valid reason to  
dnf to blow chunks is embarassing. This whole thread is mind-boggling.


I'll be more than happy to eat crow, if proven wrong by pointing out some  
detail that I missed; but if I consider anything short of a direct SIGKILL  
to a dnf process (or the entire system crashing due to power loss, etc)  
ending up with an inconsistent state, to be a BUG.


No, really, I challenge anyone to reproduce this amazing feat of tmux  
surviving an unexpected X shutdown and then tell me why dnf cannot do the  
same.




pgpYrjNRpVrWc.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Chris Murphy]

On Tue, Oct 4, 2016 at 4:20 PM, Björn Persson  wrote:
> Adam Williamson wrote:
>> If you're using Workstation, the offline update system is expressly
>> designed to minimize the likelihood of this kind of problem, so please
>> do consider using it.
>
> In CentOS or Debian I can afford to reboot for every update. With
> Fedora's rapid stream of updates that's simply not workable.

I don't know what the trigger is, but GNOME Software doesn't ask you
to do updates more than once a week. If you do 'dnf update' daily,
yeah it'll find something to update most of the time.

Once a week is still pretty frequent, but Software doesn't insist that
you install the updates once they're available and downloaded. It does
it on the next reboot which you can do whenever you'd reboot anyway.



>
>> Otherwise, at least run 'dnf update' in a VT -
>> hit ctrl-alt-f3 to get a VT console login prompt, log in, and do it
>> there.
>
> In a VT I'll often be unable to review the list of updates before
> hitting Y, as I'll only see the end of the list.

It can be piped to more.




-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Gerald B. Cox writes:

it.  Apparently I've missed something along the way because now people are  
implying that using the command line tools from within GNOME or KDE are  
dangerous.  What exactly is going on?


Somewhere along the way, it seems that quite a few sharks have been jumped  
over.


Sad.




pgp1cyZnapRoK.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Chris Murphy writes:


I don't understand how you arrive at this conclusion. dnf is sitting
on the top of a house of cards when it's running in Terminal. If
anything below it dies, dnf dies and by extension so is rpm. Could dnf
be put into it's own session or scope (whatever it's called), and


Has anyone ever heard of sigaction()? Just wondering.


would that prevent it from dying if the whole GUI died? Even if true,
how does the user know to wait XX minutes before hitting the reset
button?


I heard of a command called "ps" that might come in useful, in situations  
like that.






pgpxZwfCAVS0T.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Sam Varshavchik]

Andrew Lutomirski writes:


My point is that a lot of this exposure could be avoided.  Sure,
there's a decent chance that updating packages will crash running
programs.  But, unless one of those programs is dnf, rpm, or systemd,
that shouldn't be an excuse to blow up the whole upgrade.


I agree.

As far as I'm concerned, the only possible valid reason for the system to  
end up in an inconsistent state would be if the whole thing got SIGKILLed.


And I strongly doubt that an X crash would SIGKILL some process started by  
dnf. SIGHUP would be more likely. Perhaps SIGTERM, but neither of those  
should result in dnf blowing chunks and messing things up. Either the most  
recent transaction should be rolled back, or completed anyway.


The notion that an X crash would result in such situation is quite … 
inexplicable. There's simply no valid, logical reason for that, and I'm  
quite disappointed to hear that.



Firefox blow up many times due to concurrent dnf, but this doesn't
hose my system.  Having gnome-terminal or X or Wayland die shouldn't
be any more dangerous.


Agreed.

And even if dnf itlsef is SIGKILLed, that may certainly result in a system  
being temporarily in an inconsistent state, but I would expect that unless  
something like glibc was in the process of being unpacked, it should be  
recoverable.





pgpDGX3jAI1L8.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: ... and Fedora 25! - Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Ian Pilcher]

On 10/04/2016 01:03 PM, Tomasz Torcz wrote:

On Tue, Oct 04, 2016 at 12:31:43PM -0500, Ian Pilcher wrote:


Can you clarify?  In what circumstances would the dnf command running
within a screen session not survive an X/desktop crash?



   KillUserProcesses=yes



Ouch!  Forgot about that.

--

Ian Pilcher arequip...@gmail.com
 "I grew up before Mark Zuckerberg invented friendship" 

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Björn Persson]

stan wrote:
> On Wed, 5 Oct 2016 00:20:25 +0200
> Björn Persson  wrote:
>  
> > In a VT I'll often be unable to review the list of updates before
> > hitting Y, as I'll only see the end of the list.  
> 
> An alternative to Adam's suggestions.
> 
> It takes a couple of logins as root, but running
> dnf update > /tmp/dnf_out 2> /tmp/dnf_ror
> in one terminal and then doing a 
> less /tmp/dnf_*
> in another allows a view of the updates before approving or denying
> them. When you see the (y or N) in the less output, go back to the
> console running the program and make your selection.  A little clumsy,
> but works fine.

It's pretty obvious that improvement is needed when people start to
suggest *that* kind of hack-arounds.

Björn Persson


pgpm9eQH_INz2.pgp
Description: OpenPGP digital signatur
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Björn Persson]

Chris Murphy wrote:
> It's not really workable without an atomic and out of tree update
> method, otherwise libraries are still yanked out from under running
> processes at some point. 

Running programs and their loaded libraries count as open files. Unixy
filesystems don't delete open files. They change the directory entry to
point to the new file and keep the old file around until it's closed.
Therefore libraries are *not* yanked out from under running processes.

What tends to break is badly designed GUI programs that apparently
don't keep their files open but still expect different files to be from
the exact same version. Firefox and Seamonkey seem to do something like
that. They tend to break in funny ways when updated. Many other GUI
programs have no problems with getting updated while running.

Björn Persson


pgp3XdnO0ZVry.pgp
Description: OpenPGP digital signatur
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PSA: Do not run 'dnf update' inside GNOME, KDE or any other graphical desktop on Fedora 24

[Björn Persson]

Adam Williamson wrote:
> On Wed, 2016-10-05 at 00:20 +0200, Björn Persson wrote:
> > Adam Williamson wrote:  
> > > If you're using Workstation, the offline update system is
> > > expressly designed to minimize the likelihood of this kind of
> > > problem, so please do consider using it.  
> > 
> > 
> > In CentOS or Debian I can afford to reboot for every update. With
> > Fedora's rapid stream of updates that's simply not workable.  
> 
> Then just don't apply every update. There's no law that says you have
> to...if there's a pending security update you get a different and more
> urgent notification, btw.

I don't get any notifications at all. I just try to remember to run Yum
periodically, and there I'm not told which updates are security updates.

I used to subscribe to the package-announce list, but I had to drop that
when some broken program started sending invalid mails to the list, and
the list server kicked me out for rejecting invalid mail. It was never a
very good notification mechanism anyway, as I always had to wait a day
or two for the packages to appear on the mirrors before I could update.

Many years ago there was sometimes a tray applet that could notify me
about pending updates, but it disappeared. If something similar exists
now, then I don't know its name so I don't know what to install.
Perhaps there is some notification mechanism in Gnome 3; I wouldn't
know. I'm not aunt Tillie so I don't use Gnome 3.

Björn Persson


pgpvv1clIcwfk.pgp
Description: OpenPGP digital signatur
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org