Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[stan]

On Mon, 10 Oct 2016 15:43:04 +
John Florian  wrote:

[snip]

> Thanks for all the feedback Adam.  I'll start playing around with
> livemedia-creator to learn how my world needs to transform.  It will
> be interesting to see how this all dovetails with the stateless
> support[0] that the systemd folks have (had?) been working on.
> 
> [0] http://0pointer.net/blog/projects/stateless.html

On a tangent, reading that triggered some thoughts about how linux
systems are organized.  They seem to still be structured as if linux
was monolithic like the historical proprietary unix flavors it emulates.
It makes no sense to structure a system like that when it is composed of
changeable packages, is updated regularly, and can shrink or grow.

In combination with the concept of modularity I read about in another
email today, it would make a lot more sense to have something
like a /usr/pkgs directory where every installed package has a
directory, and put etc, share, man, bin?, lib64? ... under that
directory. i.e. every package is modular.  To satisfy traditional
software, a symbolic, or hard, link could be inserted
in /usr/bin, /usr/lib64, /etc, /usr/share/man, etc.

Just an idea, thought I'd throw it out there.  Might be completely
unworkable.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[John Florian]

On Fri, 2016-10-07 at 12:38 -0700, Adam Williamson wrote:

On Mon, 2016-10-03 at 20:03 +, John Florian wrote:


On Mon, 2016-10-03 at 12:07 -0700, Adam Williamson wrote:
If we do not 'support' livecd-iso-to-disk any more, we no longer
support:

1) persistent storage (via overlays)
2) non-destructive write


I've known for quite some time that livecd-tools was/is to be
replaced with livemedia-creator, but only now did I realize that lm-c
won't have persistent storage -- I simply have never had the time to
explore it.  I'm extremely dependent on the persistent storage as my
whole day job revolves around making hundreds of little mostly-
stateless appliances for data collection purposes and has so since
F13 or so.  These have been built with livecd-iso-to-disk and lots of
glue via specialized kickstarts and other custom packages.  These
appliances leverage a stateless OS very robustness, but do expect
some persistent storage for their management.  So the above certainly
caught my attention.



There's a slight misconception in the above.

livemedia-creator *creates the image files themselves*. We're not
talking about that in this thread. We're talking about the tools for
taking an image that's been created - whether by livecd-creator or
livemedia-creator or anything else - and writing them to a USB stick.


I'm fine with that and I think it matches my understanding/expectations.  Do 
one thing and do it we



The 'persistence' feature requires support both in the image itself and
in the tool used to write it. I believe livemedia-creator-produced
images are set up to support persistence, just like livecd-creator-
produced images were.


I suspect this "support within the image" overlaps with some of the glue to 
which I referred.  Once upon a time these live images auto-mounted their 
backing storage (where the immutable squashfs image is kept).  Then that 
stopped working and I had to write an init service to do the same.  Whether 
that service is still needed or not, I haven't investigated -- it's been 
working happily.



The issue here is that we are discussing what tools for *writing the
image to a USB stick* should be 'supported' / 'recommended' / whatever,
and we'd kinda like to drop livecd-iso-to-disk from that group, but it
is currently the only one of the 'write image to stick' tools which
supports persistence.

No-one's proposing dropping livecd-iso-to-disk entirely at present, so
you will still be able to attempt to write sticks with persistent
storage, but we are discussing effectively a downgrade in how much
testing it gets and how much we care if it's broken.


I'm fine with all that.  I certainly don't expect Fedora to cater to my 
specific needs, but I like to chime in now and then if it might help preserve 
some feature that everyone might otherwise feel is unused.



It is worth noting that we've never formally tested the persistence
features in any case, so we would never have blocked a release for
'persistence doesn't work right' anyhow. But at present we do, by
policy, block the release if writing sticks with livecd-iso-to-disk
doesn't work.


No worries there.  I/we've been testing it exhaustively.  When I push an update 
to hundreds a machines (most of which are headless/keyboard-less and ill-suited 
for such attachments and work) that all deploy it autonomously, it has to "just 
work".  Though I would never suggest our mutation reflects what the real Fedora 
is doing.  [:-)] ​





Are there plans to get persistent storage capabilities into lm-c?

Also, after much work I managed to get my live ISO spins generated
out of a private Koji setup.  I see there a warning "spin-livecd is
deprecated and will be replaced with spin-livemedia" -- I assume this
related, true?  If so, do any improvements to lm-c (say to add
persistence) automagically benefit the "spin-livemedia" method in
Koji?



Well, yeah. 'spin-livecd' is the Koji method for creating images with
livecd-creator; it's now deprecated and never used in the official
Fedora Koji instance, Fedora live images are all now created with the
'spin-livemedia' method. 'spin-livemedia' is the Koji method for
creating images with livemedia-creator.



So since what 'spin-livemedia' *does* is create a live image using
livemedia-creator, of course any changes to livemedia-creator will be
reflected when you create an image with the Koji 'spin-livemedia'
method.


Thanks for all the feedback Adam.  I'll start playing around with 
livemedia-creator to learn how my world needs to transform.  It will be 
interesting to see how this all dovetails with the stateless support[0] that 
the systemd folks have (had?) been working on.

[0] http://0pointer.net/blog/projects/stateless.html

--

John Florian >
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Sat, 2016-10-08 at 22:33 +, Zbigniew Jędrzejewski-Szmek wrote:
> On Sat, Oct 08, 2016 at 08:59:33AM -0700, Adam Williamson wrote:
> > On Sat, 2016-10-08 at 14:24 +, Zbigniew Jędrzejewski-Szmek wrote:
> > > On Mon, Oct 03, 2016 at 12:49:07PM -0600, Chris Murphy wrote:
> > > > Based on today's blocker review meeting discussion, and this email
> > > > thread [1] I'd like to propose making only Fedora Media Writer the
> > > > *officially supported* USB installation media creation tool, starting
> > > > with Fedora 26.
> > > 
> > > 
> > > 
> > > First question: is FMW the same thing as liveusb-creator? In an
> > > up-to-date F25 machine, searching for "media", "fedora", and "writer"
> > > in the gnome search dialogue yields no results. A few simple searches
> > > live 'dnf search fedora-media-writer' and 'dnf search fmw' also yield
> > > nothing. How are people supposed to find this?
> > 
> > 
> > It's 'mediawriter', if you have it installed, searching for 'media' in
> > GNOME Shell should definitely be showing it.
> 
> Yep, once it's installed. Before installation, gnome shell shows "no results"
> for "mediawriter".
> 
> OK, so the whole thing is very confusing. Google for "fedora media writer"
> returns here:
> 1. 
> http://developers.redhat.com/blog/2016/04/26/fedora-media-writer-the-fastest-way-to-create-live-usb-boot-media/
> 
> which says "sudo dnf install liveusb-creator"

Yeah, that was the *first* rewrite, when it got sort-of renamed but the
package name remained the same. The *second* rewrite happened between
April and now and resulted in the new package.

> 3. https://fedoraproject.org/wiki/Test_Day:2016-04-19_Fedora_Media_Writer
> 
> which says "sudo dnf --enablerepo=updates-testing --refresh --best install 
> liveusb-creator"

Ditto, that was the test day from April, when the package rename had
not yet happened.

> Positions 2. and 4. are about mediawriter, 5. is again about liveusb-creator,
> then a bunch is about mediawriter. Maybe it would be reasonable to add a big
> banner on pages 1. and 3. to use mediawriter instead.

Well, they're both kinda transient pages, I'd expect that over time
they'll simply naturally drop out of the top of the Google results. We
have lots and lots of old Test Day pages with no-longer-current data in
them, it'd be a full-time job going through them all and sticking
banners in them where they're no longer accurate...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Sat, 2016-10-08 at 11:40 -0600, Chris Murphy wrote:
> On Sat, Oct 8, 2016 at 8:24 AM, Zbigniew Jędrzejewski-Szmek
>  wrote:
> > On Mon, Oct 03, 2016 at 12:49:07PM -0600, Chris Murphy wrote:
> > > Based on today's blocker review meeting discussion, and this email
> > > thread [1] I'd like to propose making only Fedora Media Writer the
> > > *officially supported* USB installation media creation tool, starting
> > > with Fedora 26.
> > 
> > 
> > First question: is FMW the same thing as liveusb-creator?
> 
> 
> No, it's a rewrite. But there is confusion about handling obsoleting
> LUC or keeping it around somehow because it does support persistence.
> The last I heard the consensus was to obsolete it. Plus I think it's
> kinda broken at the moment, so how to support multiple writers? It's
> difficult enough as it is, which is the point of this thread.
> 
> 
> > In an
> > up-to-date F25 machine, searching for "media", "fedora", and "writer"
> > in the gnome search dialogue yields no results. A few simple searches
> > live 'dnf search fedora-media-writer' and 'dnf search fmw' also yield
> > nothing. How are people supposed to find this?
> 
> 
> So long as it passes QA testing, which it has so far, the plan is for
> it to be the primary downloadable. Instead of getting an ISO, you get
> a platform specific copy of Fedora Media Writer. The tool downloads
> the Fedora image of choice, checks the hash, and writes it to a stick.
> 
> But yeah finding the *current* version right now is a bit difficult.
> 
> https://fedoraproject.org/wiki/QA:Testcase_USB_fmw suggests:
> 
> sudo dnf --enablerepo=updates-testing --refresh --best install mediawriter

The --enablerepo=updates-testing should now only be necessary for F23,
a good build is stable for F24 and F25.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Zbigniew Jędrzejewski-Szmek]

On Sat, Oct 08, 2016 at 08:59:33AM -0700, Adam Williamson wrote:
> On Sat, 2016-10-08 at 14:24 +, Zbigniew Jędrzejewski-Szmek wrote:
> > On Mon, Oct 03, 2016 at 12:49:07PM -0600, Chris Murphy wrote:
> > > Based on today's blocker review meeting discussion, and this email
> > > thread [1] I'd like to propose making only Fedora Media Writer the
> > > *officially supported* USB installation media creation tool, starting
> > > with Fedora 26.
> > 
> > 
> > First question: is FMW the same thing as liveusb-creator? In an
> > up-to-date F25 machine, searching for "media", "fedora", and "writer"
> > in the gnome search dialogue yields no results. A few simple searches
> > live 'dnf search fedora-media-writer' and 'dnf search fmw' also yield
> > nothing. How are people supposed to find this?
> 
> It's 'mediawriter', if you have it installed, searching for 'media' in
> GNOME Shell should definitely be showing it.
Yep, once it's installed. Before installation, gnome shell shows "no results"
for "mediawriter".

OK, so the whole thing is very confusing. Google for "fedora media writer"
returns here:
1. 
http://developers.redhat.com/blog/2016/04/26/fedora-media-writer-the-fastest-way-to-create-live-usb-boot-media/

which says "sudo dnf install liveusb-creator"

3. https://fedoraproject.org/wiki/Test_Day:2016-04-19_Fedora_Media_Writer

which says "sudo dnf --enablerepo=updates-testing --refresh --best install 
liveusb-creator"

Positions 2. and 4. are about mediawriter, 5. is again about liveusb-creator,
then a bunch is about mediawriter. Maybe it would be reasonable to add a big
banner on pages 1. and 3. to use mediawriter instead.

> > Second question: how do I run this:
> > 
> > Clicking on the liveusbcreator icon gives me a polkit dialogue and a
> > spinning cursor which disappears after a while with no futher effect.
> 
> The liveusb-creator package still exists but is badly broken. I already
> filed a bug suggesting mediawriter should obsolete it.

OK, indeed, it seems to run nicely on F25.

Unfortunately it still dumps core if it cannot connect to a display
(https://bugzilla.redhat.com/show_bug.cgi?id=1382990).

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

On Sat, Oct 8, 2016 at 8:24 AM, Zbigniew Jędrzejewski-Szmek
 wrote:
> On Mon, Oct 03, 2016 at 12:49:07PM -0600, Chris Murphy wrote:
>> Based on today's blocker review meeting discussion, and this email
>> thread [1] I'd like to propose making only Fedora Media Writer the
>> *officially supported* USB installation media creation tool, starting
>> with Fedora 26.
>
> First question: is FMW the same thing as liveusb-creator?

No, it's a rewrite. But there is confusion about handling obsoleting
LUC or keeping it around somehow because it does support persistence.
The last I heard the consensus was to obsolete it. Plus I think it's
kinda broken at the moment, so how to support multiple writers? It's
difficult enough as it is, which is the point of this thread.


> In an
> up-to-date F25 machine, searching for "media", "fedora", and "writer"
> in the gnome search dialogue yields no results. A few simple searches
> live 'dnf search fedora-media-writer' and 'dnf search fmw' also yield
> nothing. How are people supposed to find this?

So long as it passes QA testing, which it has so far, the plan is for
it to be the primary downloadable. Instead of getting an ISO, you get
a platform specific copy of Fedora Media Writer. The tool downloads
the Fedora image of choice, checks the hash, and writes it to a stick.

But yeah finding the *current* version right now is a bit difficult.

https://fedoraproject.org/wiki/QA:Testcase_USB_fmw suggests:

sudo dnf --enablerepo=updates-testing --refresh --best install mediawriter



-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Sat, 2016-10-08 at 14:24 +, Zbigniew Jędrzejewski-Szmek wrote:
> On Mon, Oct 03, 2016 at 12:49:07PM -0600, Chris Murphy wrote:
> > Based on today's blocker review meeting discussion, and this email
> > thread [1] I'd like to propose making only Fedora Media Writer the
> > *officially supported* USB installation media creation tool, starting
> > with Fedora 26.
> 
> 
> First question: is FMW the same thing as liveusb-creator? In an
> up-to-date F25 machine, searching for "media", "fedora", and "writer"
> in the gnome search dialogue yields no results. A few simple searches
> live 'dnf search fedora-media-writer' and 'dnf search fmw' also yield
> nothing. How are people supposed to find this?

It's 'mediawriter', if you have it installed, searching for 'media' in
GNOME Shell should definitely be showing it.

> Second question: how do I run this:
> 
> Clicking on the liveusbcreator icon gives me a polkit dialogue and a
> spinning cursor which disappears after a while with no futher effect.

The liveusb-creator package still exists but is badly broken. I already
filed a bug suggesting mediawriter should obsolete it.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Zbigniew Jędrzejewski-Szmek]

On Mon, Oct 03, 2016 at 12:49:07PM -0600, Chris Murphy wrote:
> Based on today's blocker review meeting discussion, and this email
> thread [1] I'd like to propose making only Fedora Media Writer the
> *officially supported* USB installation media creation tool, starting
> with Fedora 26.

First question: is FMW the same thing as liveusb-creator? In an
up-to-date F25 machine, searching for "media", "fedora", and "writer"
in the gnome search dialogue yields no results. A few simple searches
live 'dnf search fedora-media-writer' and 'dnf search fmw' also yield
nothing. How are people supposed to find this?

Second question: how do I run this:

Clicking on the liveusbcreator icon gives me a polkit dialogue and a
spinning cursor which disappears after a while with no futher effect.

$ liveusb-creator
You must run this application as root
[1]

$ sudo liveusb-creator
No protocol specified
QXcbConnection: Could not connect to display :0
[2]11324 abort  sudo liveusb-creator

$ liveusb-creator_polkit
Invalid MIT-MAGIC-COOKIE-1 keyQXcbConnection: Could not connect to display :0
/usr/bin/liveusb-creator_polkit: line 6: 11411 Aborted (core dumped) pkexec 
--disable-internal-agent "/usr/bin/liveusb-creator" "$@"
[134]

(gdb) bt
#0  0x7fb1e1f6a92f in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:58
#1  0x7fb1e1f6c52a in __GI_abort () at abort.c:89
#2  0x7fb1da2baaf1 in  () at /lib64/libQt5Core.so.5
#3  0x7fb1cf69cb5e in QXcbConnection::QXcbConnection(QXcbNativeInterface*, 
bool, unsigned int, char const*) ()
at /lib64/libQt5XcbQpa.so.5
#4  0x7fb1cf69fd1e in QXcbIntegration::QXcbIntegration(QStringList const&, 
int&, char**) () at /lib64/libQt5XcbQpa.so.5
#5  0x7fb1e321c6ed in QXcbIntegrationPlugin::create(QString const&, 
QStringList const&, int&, char**) ()
at /usr/lib64/qt5/plugins/platforms/libqxcb.so
#6  0x7fb1d5e748fd in QPlatformIntegrationFactory::create(QString const&, 
QStringList const&, int&, char**, QString const&) ()
at /lib64/libQt5Gui.so.5
#7  0x7fb1d5e820c1 in QGuiApplicationPrivate::createPlatformIntegration() 
() at /lib64/libQt5Gui.so.5
#8  0x7fb1d5e82f3d in QGuiApplicationPrivate::createEventDispatcher() () at 
/lib64/libQt5Gui.so.5
#9  0x7fb1da4754f9 in QCoreApplicationPrivate::init() () at 
/lib64/libQt5Core.so.5
#10 0x7fb1d5e8419e in QGuiApplicationPrivate::init() () at 
/lib64/libQt5Gui.so.5
#11 0x7fb1d1ea6619 in QApplicationPrivate::init() () at 
/lib64/libQt5Widgets.so.5
#12 0x7fb1d265fb99 in sipQApplication::sipQApplication(int&, char**, int) 
() at /usr/lib64/python2.7/site-packages/PyQt5/QtWidgets.so
#13 0x7fb1d265fc73 in init_type_QApplication () at 
/usr/lib64/python2.7/site-packages/PyQt5/QtWidgets.so
#14 0x7fb1d6a4f78b in sipSimpleWrapper_init () at 
/usr/lib64/python2.7/site-packages/sip.so
#15 0x7fb1e2cc45cc in wrap_init () at /lib64/libpython2.7.so.1.0
#16 0x7fb1e2c73003 in PyObject_Call () at /lib64/libpython2.7.so.1.0
#17 0x7fb1e2d05107 in PyEval_CallObjectWithKeywords () at 
/lib64/libpython2.7.so.1.0
#18 0x7fb1e2c88cf5 in wrapperdescr_call () at /lib64/libpython2.7.so.1.0
#19 0x7fb1e2c73003 in PyObject_Call () at /lib64/libpython2.7.so.1.0
#20 0x7fb1e2d0ae66 in PyEval_EvalFrameEx () at /lib64/libpython2.7.so.1.0
#21 0x7fb1e2d0f05c in PyEval_EvalCodeEx () at /lib64/libpython2.7.so.1.0
#22 0x7fb1e2c97d6c in function_call () at /lib64/libpython2.7.so.1.0
#23 0x7fb1e2c73003 in PyObject_Call () at /lib64/libpython2.7.so.1.0
#24 0x7fb1e2c81e5c in instancemethod_call () at /lib64/libpython2.7.so.1.0
#25 0x7fb1e2c73003 in PyObject_Call () at /lib64/libpython2.7.so.1.0
#26 0x7fb1e2cca127 in slot_tp_init () at /lib64/libpython2.7.so.1.0
#27 0x7fb1e2cc8dee in type_call () at /lib64/libpython2.7.so.1.0
#28 0x7fb1e2c73003 in PyObject_Call () at /lib64/libpython2.7.so.1.0
#29 0x7fb1e2d0ae66 in PyEval_EvalFrameEx () at /lib64/libpython2.7.so.1.0
#30 0x7fb1e2d0c052 in PyEval_EvalFrameEx () at /lib64/libpython2.7.so.1.0
#31 0x7fb1e2d0f05c in PyEval_EvalCodeEx () at /lib64/libpython2.7.so.1.0
#32 0x7fb1e2d0f149 in PyEval_EvalCode () at /lib64/libpython2.7.so.1.0
#33 0x7fb1e2d2852f in run_mod () at /lib64/libpython2.7.so.1.0
#34 0x7fb1e2d29762 in PyRun_FileExFlags () at /lib64/libpython2.7.so.1.0
#35 0x7fb1e2d2a975 in PyRun_SimpleFileExFlags () at 
/lib64/libpython2.7.so.1.0
#36 0x7fb1e2d3c900 in Py_Main () at /lib64/libpython2.7.so.1.0
#37 0x7fb1e1f55401 in __libc_start_main (main=
0x563e8dd427a0 , argc=3, argv=0x7ffece2797e8, init=, 
fini=, rtld_fini=, stack_end=0x7ffece2797d8) at 
../csu/libc-start.c:289
#38 0x563e8dd427da in _start ()

Waylandly-confused,
Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Kevin Kofler]

Chris Murphy wrote:
> dmverity is a valid option geared explicitly for this and can include
> Reed-Solomon error correction;

Ouch, please no! Reed-Solomon is going to blow up the size of the live 
images a lot, when all that's really needed is a simple checksum.

> and Btrfs. The Btrfs option is kinda need because a.) it's on-the-fly
> instead of all at once b.) it's every time a block is read, not one time,
> c.) can't be bypassed by the user, and d.) Btrfs supports overlays with
> the seed device function.

That's not quite the same thing as verification of the entire media. It 
would also only be possible to replace the inner ext4 image with Btrfs, not 
the outer SquashFS one, because Btrfs natively only supports zlib or LZO 
compression, not the xz compression we are currently using. Switching (back) 
to zlib would blow up the size of the live images a lot, and LZO is even 
worse. The documentation says that LZMA/xz support is not currently being 
considered in Btrfs. Btrfs also only compresses file contents, not metadata, 
which is another reason an inner image would still be needed (to turn the 
metadata into compressible contents). And whatever you do, the size and 
performance impact of switching to Btrfs would have to be measured, it could 
be significant in either direction.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Kevin Kofler]

Andrew Lutomirski wrote:
> At the risk of muddying the waters a bit: now that OverlayFS is here, I
> think that even a dd-copied image should be able to support persistence.
> The image could notice that it's dd-copied (by checking GPT GUIDs or
> layout or whatever), see that there's extra space at the end, and allocate
> it.
> 
> This could reduce the testing explosion a bit if all of the supported
> image writing tools ended up being equivalent to dd.

It wouldn't work for the non-destructive use case.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

On Fri, Oct 7, 2016 at 5:58 PM, Andrew Lutomirski  wrote:
> On Fri, Oct 7, 2016 at 2:32 PM, Chris Murphy  wrote:
>> Modifying the image at all breaks the existing media verification
>> option in the boot menu, and we know people get bad writes using bad
>> or flaky media
>
> This part, at least, should be relatively straightforward to get
> around.  The allocation of a persistence partition literally just
> create a new partition, so if the media verification could learn to
> verify the partition (bitwise) instead of the whole device, it would
> work fine.

Maybe.

This is what we're doing to create our ISO images. Note there are
three partition maps needing modification: MBR, GPT, and APM.
https://mjg59.dreamwidth.org/4957.html

The md5's are embedded in an ISO 9660 metadata area using
implantisomd5, and rd.live.check causes dracut to run checkisomd5.
This isn't partition based. I don't see a configurable offset, but
maybe there's a fixed offset that puts the partition maps outside the
area being checked.

Anyway, it's still suboptimal:

1. On macOS, right after media creation, the OS always automounts the
HFS+ volume found on these ISOs, read-write. This instantly changes
the content of a portion of the media that is subject to media
verification. So any stick created on macOS always fails media
verification. Mbriza has said he's heard this happens on Windows also.
I haven't tested it.

2. It's slow and interrupts user flow.

3. It's one shot, not every read is checked every time.

4. The user can opt out.

For optical media, it's a good solution that's portable. Optical media
is unlikely to have transient corruptions. But I think it's inadequate
for flash media where they can produce transient corruption on reads
without any error being reported by the stick itself. So if media
verification is at all important, I think the longer term plan should
look at dmverity or Btrfs seed device.



-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Lars Seipel]

On Tue, Oct 04, 2016 at 09:16:12AM -0600, Chris Murphy wrote:
> Over on Windows and macOS, there is no such thing as a non-destructive
> install media creation. It warns but obliterates the entire stick.
> They also don't have persistence. So I think you're on very solid
> ground calling both features edge cases, and as such probably not
> reasonable to block a release on if it weren't to work.

For these systems and most of their user base, installing the OS from
scratch already is an edge case. Most people use the preinstalled copy
that came with their machine. It wasn't that long ago that you couldn't
install them from an USB stick at all but had to use some kind of
optical disc. Long into the 2000's you had to use a freakin' floppy
drive to inject storage device drivers into the Windows install process.

Fedora is not in a position to impose such ridiculous demands on its
users and I, for one, wouldn't want to even if we could.

That's not to say we should block the release for persistence-enabled
live media. I don't think we should. It's just that the Windows or MacOS
install process isn't a very useful comparison for our needs. Fedora
has a far more diverse set of use-cases than any of these systems and
thus requires much more flexibility in its installation infrastructure.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Andrew Lutomirski]

On Fri, Oct 7, 2016 at 2:32 PM, Chris Murphy  wrote:
> Modifying the image at all breaks the existing media verification
> option in the boot menu, and we know people get bad writes using bad
> or flaky media

This part, at least, should be relatively straightforward to get
around.  The allocation of a persistence partition literally just
create a new partition, so if the media verification could learn to
verify the partition (bitwise) instead of the whole device, it would
work fine.

--Andy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Fri, 2016-10-07 at 13:32 -0700, Andrew Lutomirski wrote:
> 
> At the risk of muddying the waters a bit: now that OverlayFS is here, I
> think that even a dd-copied image should be able to support persistence.
> The image could notice that it's dd-copied (by checking GPT GUIDs or layout
> or whatever), see that there's extra space at the end, and allocate it.

In fact, our cloud images do something rather like this so that when
you write them to a hard disk, they expand the root partition to fill
the available space on the disk on the initial boot.

So yeah, it seems entirely plausible that we could design a persistence
mechanism that worked that way. But the current one doesn't :)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

On Fri, Oct 7, 2016 at 2:32 PM, Andrew Lutomirski  wrote:
> On Oct 7, 2016 12:39 PM, "Adam Williamson" 
> wrote:
>>
>> On Mon, 2016-10-03 at 20:03 +, John Florian wrote:
>> > On Mon, 2016-10-03 at 12:07 -0700, Adam Williamson wrote:
>> > If we do not 'support' livecd-iso-to-disk any more, we no longer
>> > support:
>> >
>> > 1) persistent storage (via overlays)
>> > 2) non-destructive write
>> >
>> >
>> > I've known for quite some time that livecd-tools was/is to be
>> > replaced with livemedia-creator, but only now did I realize that lm-c
>> > won't have persistent storage -- I simply have never had the time to
>> > explore it.  I'm extremely dependent on the persistent storage as my
>> > whole day job revolves around making hundreds of little mostly-
>> > stateless appliances for data collection purposes and has so since
>> > F13 or so.  These have been built with livecd-iso-to-disk and lots of
>> > glue via specialized kickstarts and other custom packages.  These
>> > appliances leverage a stateless OS very robustness, but do expect
>> > some persistent storage for their management.  So the above certainly
>> > caught my attention.
>>
>> There's a slight misconception in the above.
>>
>> livemedia-creator *creates the image files themselves*. We're not
>> talking about that in this thread. We're talking about the tools for
>> taking an image that's been created - whether by livecd-creator or
>> livemedia-creator or anything else - and writing them to a USB stick.
>>
>> The 'persistence' feature requires support both in the image itself and
>> in the tool used to write it. I believe livemedia-creator-produced
>> images are set up to support persistence, just like livecd-creator-
>> produced images were.
>>
>> The issue here is that we are discussing what tools for *writing the
>> image to a USB stick* should be 'supported' / 'recommended' / whatever,
>> and we'd kinda like to drop livecd-iso-to-disk from that group, but it
>> is currently the only one of the 'write image to stick' tools which
>> supports persistence.
>
> At the risk of muddying the waters a bit: now that OverlayFS is here, I
> think that even a dd-copied image should be able to support persistence.
> The image could notice that it's dd-copied (by checking GPT GUIDs or layout
> or whatever), see that there's extra space at the end, and allocate it.
>
> This could reduce the testing explosion a bit if all of the supported image
> writing tools ended up being equivalent to dd.

It's come up, and mbriza has some ideas about how to do it safely, so
we'll see if it's worth the risks.

I'm very skeptical that persistence is a critical necessity, rather
than neat. If you need persistence, the installer should be capable of
installing to your USB stick media and making it a real installed OS
seeing as that's what it's being used for.

Modifying the image at all breaks the existing media verification
option in the boot menu, and we know people get bad writes using bad
or flaky media. We could get this back various ways: embedded sha1sum
of the squashfs.img and have dracut do a comparison at boot time;
dmverity is a valid option geared explicitly for this and can include
Reed-Solomon error correction; and Btrfs. The Btrfs option is kinda
need because a.) it's on-the-fly instead of all at once b.) it's every
time a block is read, not one time, c.) can't be bypassed by the user,
and d.) Btrfs supports overlays with the seed device function. [1]
Seed device was actually intended for this specific use case. And it's
not a new feature, January 2009, upstream considers it stable. [2]


[1]
https://btrfs.wiki.kernel.org/index.php/Seed-device

[2]
https://btrfs.wiki.kernel.org/index.php/Status


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Andrew Lutomirski]

On Oct 7, 2016 12:39 PM, "Adam Williamson" 
wrote:
>
> On Mon, 2016-10-03 at 20:03 +, John Florian wrote:
> > On Mon, 2016-10-03 at 12:07 -0700, Adam Williamson wrote:
> > If we do not 'support' livecd-iso-to-disk any more, we no longer
> > support:
> >
> > 1) persistent storage (via overlays)
> > 2) non-destructive write
> >
> >
> > I've known for quite some time that livecd-tools was/is to be
> > replaced with livemedia-creator, but only now did I realize that lm-c
> > won't have persistent storage -- I simply have never had the time to
> > explore it.  I'm extremely dependent on the persistent storage as my
> > whole day job revolves around making hundreds of little mostly-
> > stateless appliances for data collection purposes and has so since
> > F13 or so.  These have been built with livecd-iso-to-disk and lots of
> > glue via specialized kickstarts and other custom packages.  These
> > appliances leverage a stateless OS very robustness, but do expect
> > some persistent storage for their management.  So the above certainly
> > caught my attention.
>
> There's a slight misconception in the above.
>
> livemedia-creator *creates the image files themselves*. We're not
> talking about that in this thread. We're talking about the tools for
> taking an image that's been created - whether by livecd-creator or
> livemedia-creator or anything else - and writing them to a USB stick.
>
> The 'persistence' feature requires support both in the image itself and
> in the tool used to write it. I believe livemedia-creator-produced
> images are set up to support persistence, just like livecd-creator-
> produced images were.
>
> The issue here is that we are discussing what tools for *writing the
> image to a USB stick* should be 'supported' / 'recommended' / whatever,
> and we'd kinda like to drop livecd-iso-to-disk from that group, but it
> is currently the only one of the 'write image to stick' tools which
> supports persistence.

At the risk of muddying the waters a bit: now that OverlayFS is here, I
think that even a dd-copied image should be able to support persistence.
The image could notice that it's dd-copied (by checking GPT GUIDs or layout
or whatever), see that there's extra space at the end, and allocate it.

This could reduce the testing explosion a bit if all of the supported image
writing tools ended up being equivalent to dd.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

On Fri, Oct 7, 2016 at 1:26 PM, Adam Williamson
 wrote:
> On Tue, 2016-10-04 at 09:16 -0600, Chris Murphy wrote:
> The one concern I have is with Sugar on a Stick spin. Their
> installation instructions require livecd-iso-to-disk, because their
> media boots straight into SoaS, not Anaconda. But I have some ideas
> about how to deal with that going forward to, rather than depend
> indefinitely on livecd-iso-to-disk.
>
> I don't quite get what you mean by this. Are you saying that they
> recommend using livecd-iso-to-disk with persistence as the official
> method for 'installing' SoaS?

They recommend various methods but yes...

https://wiki.sugarlabs.org/go/Sugar_on_a_Stick/Installation

In the Download section you get different instructions, to use Live
USB Creator with the persistence slider set. And same for Unetbootin.
In all cases, it's copy the files in the SoaS ISO to the stick, and
setup persistence. There's no use of Anaconda.



-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Mon, 2016-10-03 at 20:03 +, John Florian wrote:
> On Mon, 2016-10-03 at 12:07 -0700, Adam Williamson wrote:
> If we do not 'support' livecd-iso-to-disk any more, we no longer
> support:
> 
> 1) persistent storage (via overlays)
> 2) non-destructive write
> 
> 
> I've known for quite some time that livecd-tools was/is to be
> replaced with livemedia-creator, but only now did I realize that lm-c 
> won't have persistent storage -- I simply have never had the time to
> explore it.  I'm extremely dependent on the persistent storage as my
> whole day job revolves around making hundreds of little mostly-
> stateless appliances for data collection purposes and has so since
> F13 or so.  These have been built with livecd-iso-to-disk and lots of
> glue via specialized kickstarts and other custom packages.  These
> appliances leverage a stateless OS very robustness, but do expect
> some persistent storage for their management.  So the above certainly
> caught my attention.

There's a slight misconception in the above.

livemedia-creator *creates the image files themselves*. We're not
talking about that in this thread. We're talking about the tools for
taking an image that's been created - whether by livecd-creator or
livemedia-creator or anything else - and writing them to a USB stick.

The 'persistence' feature requires support both in the image itself and
in the tool used to write it. I believe livemedia-creator-produced
images are set up to support persistence, just like livecd-creator-
produced images were.

The issue here is that we are discussing what tools for *writing the
image to a USB stick* should be 'supported' / 'recommended' / whatever,
and we'd kinda like to drop livecd-iso-to-disk from that group, but it
is currently the only one of the 'write image to stick' tools which
supports persistence.

No-one's proposing dropping livecd-iso-to-disk entirely at present, so
you will still be able to attempt to write sticks with persistent
storage, but we are discussing effectively a downgrade in how much
testing it gets and how much we care if it's broken.

It is worth noting that we've never formally tested the persistence
features in any case, so we would never have blocked a release for
'persistence doesn't work right' anyhow. But at present we do, by
policy, block the release if writing sticks with livecd-iso-to-disk
doesn't work.

> Are there plans to get persistent storage capabilities into lm-c?
> 
> Also, after much work I managed to get my live ISO spins generated
> out of a private Koji setup.  I see there a warning "spin-livecd is
> deprecated and will be replaced with spin-livemedia" -- I assume this
> related, true?  If so, do any improvements to lm-c (say to add
> persistence) automagically benefit the "spin-livemedia" method in
> Koji?

Well, yeah. 'spin-livecd' is the Koji method for creating images with
livecd-creator; it's now deprecated and never used in the official
Fedora Koji instance, Fedora live images are all now created with the
'spin-livemedia' method. 'spin-livemedia' is the Koji method for
creating images with livemedia-creator.

So since what 'spin-livemedia' *does* is create a live image using
livemedia-creator, of course any changes to livemedia-creator will be
reflected when you create an image with the Koji 'spin-livemedia'
method.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Wed, 2016-10-05 at 21:51 +0200, Kevin Kofler wrote:
> I personally think that wiping all existing data on the USB stick is 
> extremely user-unfriendly. A non-destructive method that just works would 
> make everyone happy (both the majority that wants something that just works, 
> no matter how, and those who want the non-destructive method). It is sad 
> that liveusb-creator chickened out and implemented the dd option instead of 
> fixing the issues with the non-destructive method, and that the rewrite is 
> even destructive only. The wasted space due to the lack of a persistent 
> overlay is also sad.

The problem is that it's very difficult to do *all of the following*:

1. Write the stick non-destructively
2. Ensure it boots on as many systems as possible via BIOS
3. Ensure it boots on as many systems as possible via UEFI
4. Ensure it boots on Macs (a special case of 3, basically)

there are all kinds of considerations in achieving 2, 3 and 4 with a
single stick, and trying to achieve 1 as well immediately makes 2, 3
and 4 massively harder. Not caring about 1 makes 2, 3 and 4 a lot
easier.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Tue, 2016-10-04 at 09:16 -0600, Chris Murphy wrote:
The one concern I have is with Sugar on a Stick spin. Their
installation instructions require livecd-iso-to-disk, because their
media boots straight into SoaS, not Anaconda. But I have some ideas
about how to deal with that going forward to, rather than depend
indefinitely on livecd-iso-to-disk.

I don't quite get what you mean by this. Are you saying that they
recommend using livecd-iso-to-disk with persistence as the official
method for 'installing' SoaS?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Tue, 2016-10-04 at 11:12 +0200, Martin Kolman wrote:
> 
> > There is currently no real way to use FMW on non-Fedora Linux
> > distributions that don't a) support Flatpak and b) have an
> > appropriate
> > Flatpak runtime for running FMW on (beyond compiling it yourself, I
> > guess).
> 
> That sounds weird - why can't it be packaged for other distros by using
> the normal distro packaging mechanisms (RPM/deb,etc. packages) ?

There's no reason it *couldn't* be done, it just *hasn't* been done.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Kevin Kofler]

Adam Williamson wrote:
> It's simply the case that no-one's cared enough about those features to
> rewrite them. A few people have kicked around ideas for how to do it at
> various points, but it's never gone beyond that.

I think the main issue there is that the use cases for live USB media have 
changed. In the past, it used to be that the standard way to get live media 
was to burn a CD or DVD, and that those who wanted USB wanted it exactly FOR 
those extra features (in particular, the persistent overlay). But these 
days, more and more people don't have optical drives at all, and so want USB 
just to get their live image booting at all, they don't care how. Thus, the 
extra features, which used to be the whole point of making live USB sticks, 
have suddenly become niche features.

That doesn't mean the demand has disappeared. It is just eclipsed by the 
mass of users who were previously burning DVDs or even CDs.

I personally think that wiping all existing data on the USB stick is 
extremely user-unfriendly. A non-destructive method that just works would 
make everyone happy (both the majority that wants something that just works, 
no matter how, and those who want the non-destructive method). It is sad 
that liveusb-creator chickened out and implemented the dd option instead of 
fixing the issues with the non-destructive method, and that the rewrite is 
even destructive only. The wasted space due to the lack of a persistent 
overlay is also sad.

It is also not a valid assumption that everyone has USB sticks to spare.

I also don't understand Kamil Paral's point:
> It almost never works for standard users, unless you have a very good
> understanding what a bootloader is and whether you should replace it or
> not.
as we can handle this issue just fine for hard disk installations.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

On Tue, Oct 4, 2016 at 1:46 AM, Kamil Paral  wrote:
>> > If we do not 'support' livecd-iso-to-disk any more, we no longer
>> > support:
>> >
>> > 1) persistent storage (via overlays)
>> > 2) non-destructive write
>>
>> Does anyone know why we can't have Fedora Media Writer support these
>> functions as well?
>
> I hope it won't. Or if it will, I hope it won't be the default, it will be 
> well hidden, and we won't block on it. Because especially the non-destructive 
> write is a can of worms. It almost never works for standard users, unless you 
> have a very good understanding what a bootloader is and whether you should 
> replace it or not. Most people don't need it (everyone has a small flash 
> drive to be completely overwritten these days), and those who do, they can 
> easily use livecd-iso-to-disk with its heap of magical cmdline switches.
>
> So no, I don't see why our default tool which is intended to be simple and 
> user friendly should support alternative modes of operations for <1% of our 
> user base. Do one thing and do it well.

Over on Windows and macOS, there is no such thing as a non-destructive
install media creation. It warns but obliterates the entire stick.
They also don't have persistence. So I think you're on very solid
ground calling both features edge cases, and as such probably not
reasonable to block a release on if it weren't to work.

I think both features are better applied to the installer. Any
installer deficiencies installing to removable media should be
addressed. This would provide both of the listed functions. In theory
it should just work anyway.

The one concern I have is with Sugar on a Stick spin. Their
installation instructions require livecd-iso-to-disk, because their
media boots straight into SoaS, not Anaconda. But I have some ideas
about how to deal with that going forward to, rather than depend
indefinitely on livecd-iso-to-disk.



-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Michael Catanzaro]

On Tue, 2016-10-04 at 03:35 -0400, Kamil Paral wrote:
> Personally I'd block on FMW *and* dd.

Yes, explicit +1 from me. dd needs to work.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Ms Sanchez]

Agree with Kamil.  Do one thing and do it well.


Cheers,

Sylvia


On 04/10/16 09:46, Kamil Paral wrote:

If we do not 'support' livecd-iso-to-disk any more, we no longer
support:

1) persistent storage (via overlays)
2) non-destructive write

Does anyone know why we can't have Fedora Media Writer support these
functions as well?

I hope it won't. Or if it will, I hope it won't be the default, it will be well 
hidden, and we won't block on it. Because especially the non-destructive write 
is a can of worms. It almost never works for standard users, unless you have a 
very good understanding what a bootloader is and whether you should replace it 
or not. Most people don't need it (everyone has a small flash drive to be 
completely overwritten these days), and those who do, they can easily use 
livecd-iso-to-disk with its heap of magical cmdline switches.

So no, I don't see why our default tool which is intended to be simple and user 
friendly should support alternative modes of operations for <1% of our user 
base. Do one thing and do it well.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Martin Kolman]

On Mon, 2016-10-03 at 12:07 -0700, Adam Williamson wrote:
> On Mon, 2016-10-03 at 12:49 -0600, Chris Murphy wrote:
> > 
> > Based on today's blocker review meeting discussion, and this email
> > thread [1] I'd like to propose making only Fedora Media Writer the
> > *officially supported* USB installation media creation tool,
> > starting
> > with Fedora 26.
> > 
> > The practical implication of "officially support" means bugs for
> > which
> > we'd block the release. It doesn't make sense to block the release
> > if
> > myriad tools all don't succeed. We only really need one to work,
> > and
> > Fedora Media Writer is the cross platform tool we're investing in
> > long
> > term.
> > 
> > The main idea of the proposal is to no longer block the release
> > when
> > Fedora Media Writer is working, but some other possibly useful ways
> > of
> > creating media aren't working. It doesn't mean those tools won't be
> > fixed, or would be removed from the distribution, just means we're
> > not
> > holding up release for those alternative tools.
> > 
> > Comments?
> 
> There is currently no real way to use FMW on non-Fedora Linux
> distributions that don't a) support Flatpak and b) have an
> appropriate
> Flatpak runtime for running FMW on (beyond compiling it yourself, I
> guess).
That sounds weird - why can't it be packaged for other distros by using
the normal distro packaging mechanisms (RPM/deb,etc. packages) ?
> 
> If we do not 'support' livecd-iso-to-disk any more, we no longer
> support:
> 
> 1) persistent storage (via overlays)
> 2) non-destructive write
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Tue, 2016-10-04 at 03:50 -0400, Neal Gompa wrote:
> On Tue, Oct 4, 2016 at 3:46 AM, Kamil Paral  wrote:
> > If we do not 'support' livecd-iso-to-disk any more, we no longer
> > support:
> > 
> > 1) persistent storage (via overlays)
> > 2) non-destructive write
> 
> 
> Does anyone know why we can't have Fedora Media Writer support these
> functions as well?
> 
> 
> I hope it won't. Or if it will, I hope it won't be the default, it will be 
> well hidden, and we won't block on it. Because especially the non-destructive 
> write is a can of worms. It almost never works for standard users, unless you 
> have a very good understanding what a bootloader is and whether you should 
> replace it or not. Most people don't need it (everyone has a small flash 
> drive to be completely overwritten these days), and those who do, they can 
> easily use livecd-iso-to-disk with its heap of magical cmdline switches.
> 
> 
> That sounds more like we should revisit how we do those features
> rather than anything else. When livecd-iso-to-disk was created, we
> didn't have things like OverlayFS that may potentially simplify how we
> support this greatly. Most flash drives I've seen in use are at least
> 2x larger than the Fedora images, so it's almost wasteful that we
> can't leverage that extra space.

It's simply the case that no-one's cared enough about those features to
rewrite them. A few people have kicked around ideas for how to do it at
various points, but it's never gone beyond that.

There are other bits of the live image infrastructure that are severely
outdated too, and which no-one has chosen (or been paid) to modernize;
the obvious one is the way we do a lot of customization of the live
environment, by creating a couple of sysv services on the fly in the
%post sections of the live image kickstarts. The kickstarts in general
have a whole pile of junk in them.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Neal Gompa]

On Tue, Oct 4, 2016 at 3:46 AM, Kamil Paral  wrote:
>> > If we do not 'support' livecd-iso-to-disk any more, we no longer
>> > support:
>> >
>> > 1) persistent storage (via overlays)
>> > 2) non-destructive write
>>
>> Does anyone know why we can't have Fedora Media Writer support these
>> functions as well?
>
> I hope it won't. Or if it will, I hope it won't be the default, it will be 
> well hidden, and we won't block on it. Because especially the non-destructive 
> write is a can of worms. It almost never works for standard users, unless you 
> have a very good understanding what a bootloader is and whether you should 
> replace it or not. Most people don't need it (everyone has a small flash 
> drive to be completely overwritten these days), and those who do, they can 
> easily use livecd-iso-to-disk with its heap of magical cmdline switches.

That sounds more like we should revisit how we do those features
rather than anything else. When livecd-iso-to-disk was created, we
didn't have things like OverlayFS that may potentially simplify how we
support this greatly. Most flash drives I've seen in use are at least
2x larger than the Fedora images, so it's almost wasteful that we
can't leverage that extra space.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Kamil Paral]

> > If we do not 'support' livecd-iso-to-disk any more, we no longer
> > support:
> >
> > 1) persistent storage (via overlays)
> > 2) non-destructive write
> 
> Does anyone know why we can't have Fedora Media Writer support these
> functions as well? 

I hope it won't. Or if it will, I hope it won't be the default, it will be well 
hidden, and we won't block on it. Because especially the non-destructive write 
is a can of worms. It almost never works for standard users, unless you have a 
very good understanding what a bootloader is and whether you should replace it 
or not. Most people don't need it (everyone has a small flash drive to be 
completely overwritten these days), and those who do, they can easily use 
livecd-iso-to-disk with its heap of magical cmdline switches.

So no, I don't see why our default tool which is intended to be simple and user 
friendly should support alternative modes of operations for <1% of our user 
base. Do one thing and do it well.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Kamil Paral]

> Based on today's blocker review meeting discussion, and this email
> thread [1] I'd like to propose making only Fedora Media Writer the
> *officially supported* USB installation media creation tool, starting
> with Fedora 26.
> 
> The practical implication of "officially support" means bugs for which
> we'd block the release. It doesn't make sense to block the release if
> myriad tools all don't succeed. We only really need one to work, and
> Fedora Media Writer is the cross platform tool we're investing in long
> term.
> 
> The main idea of the proposal is to no longer block the release when
> Fedora Media Writer is working, but some other possibly useful ways of
> creating media aren't working. It doesn't mean those tools won't be
> fixed, or would be removed from the distribution, just means we're not
> holding up release for those alternative tools.
> 
> Comments?

Personally I'd block on FMW *and* dd. FMW uses dd-like approach internally, so 
if FMW works, dd should work as well. I'd even say that if you've successfully 
tested FMW, you can mark dd testcase passed as well. But we should still 
officially block on dd, because a) it's a universal approach, available in any 
distribution or even OS (direct copy writers exist for Windows and Mac) b) it's 
cmdline and therefore can be scripted/automated.

gnome-disks also use dd-like approach, so if dd works, gnome-disks should work 
as well modulo UI (or selinux) bugs. I'd probably not block on gnome-disks, 
because FMW does a better job as a GUI-based writer, and I don't see any extra 
benefit in gnome-disks over FMW. The only difference is that it's available in 
all distributions unlike FMW, but most probably in different versions than we 
currently have in Fedora, so blocking on it in Fedora (where we have FMW 
available) doesn't make much sense to me.

livecd-iso-to-disk seems to provide too much of an edge case functionality, so 
I wouldn't block on it either. I'm aware that persistence and non-destructive 
write is useful for some people, and this doesn't mean it will break and not 
get fixed. However, this is for a very narrow audience and I don't think 
holding up the whole release (and spending time testing it with every compose) 
is really worth it.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Lars Seipel]

On Mon, Oct 03, 2016 at 12:49:07PM -0600, Chris Murphy wrote:
> Based on today's blocker review meeting discussion, and this email
> thread [1] I'd like to propose making only Fedora Media Writer the
> *officially supported* USB installation media creation tool, starting
> with Fedora 26.

Writing the image with dd/cat/cp/whatever is probably just going to work
(if LMC does) and is available anywhere you can put it an USB stick.

I'd like that to be supported, too, so that putting workarounds for
broken images into the writing program is not sufficient to go on with
the release.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

On Mon, Oct 3, 2016 at 1:28 PM, Neal Gompa  wrote:
> On Mon, Oct 3, 2016 at 3:07 PM, Adam Williamson
>  wrote:
>>
>> If we do not 'support' livecd-iso-to-disk any more, we no longer
>> support:
>>
>> 1) persistent storage (via overlays)
>> 2) non-destructive write
>
> Does anyone know why we can't have Fedora Media Writer support these
> functions as well?

I know that 2 is planned, but I don't know the time frame.

1 in its current form is considered fragile, the l-i-t-d documentation
warns that it's expected to blowup without warning and be unfixable,
it's just a matter of time. Mbriza and I have several ideas how this
could work better, but I also don't know the time frame. A lot depends
on resources.

Also note that both of these l-i-t-d creation methods break media
verification. Media verification is the default option on Lives, but
when media is created either with persistent overlay or
non-destructively, the verification is silently skipped.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[John Florian]

On Mon, 2016-10-03 at 12:07 -0700, Adam Williamson wrote:
If we do not 'support' livecd-iso-to-disk any more, we no longer
support:

1) persistent storage (via overlays)
2) non-destructive write


I've known for quite some time that livecd-tools was/is to be replaced with 
livemedia-creator, but only now did I realize that lm-c won't have persistent 
storage -- I simply have never had the time to explore it.  I'm extremely 
dependent on the persistent storage as my whole day job revolves around making 
hundreds of little mostly-stateless appliances for data collection purposes and 
has so since F13 or so.  These have been built with livecd-iso-to-disk and lots 
of glue via specialized kickstarts and other custom packages.  These appliances 
leverage a stateless OS very robustness, but do expect some persistent storage 
for their management.  So the above certainly caught my attention.

Are there plans to get persistent storage capabilities into lm-c?

Also, after much work I managed to get my live ISO spins generated out of a 
private Koji setup.  I see there a warning "spin-livecd is deprecated and will 
be replaced with spin-livemedia" -- I assume this related, true?  If so, do any 
improvements to lm-c (say to add persistence) automagically benefit the 
"spin-livemedia" method in Koji?


--

John Florian >
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

On Mon, Oct 3, 2016 at 1:07 PM, Adam Williamson
 wrote:
> On Mon, 2016-10-03 at 12:49 -0600, Chris Murphy wrote:
>> Based on today's blocker review meeting discussion, and this email
>> thread [1] I'd like to propose making only Fedora Media Writer the
>> *officially supported* USB installation media creation tool, starting
>> with Fedora 26.
>>
>> The practical implication of "officially support" means bugs for which
>> we'd block the release. It doesn't make sense to block the release if
>> myriad tools all don't succeed. We only really need one to work, and
>> Fedora Media Writer is the cross platform tool we're investing in long
>> term.
>>
>> The main idea of the proposal is to no longer block the release when
>> Fedora Media Writer is working, but some other possibly useful ways of
>> creating media aren't working. It doesn't mean those tools won't be
>> fixed, or would be removed from the distribution, just means we're not
>> holding up release for those alternative tools.
>>
>> Comments?
>
> There is currently no real way to use FMW on non-Fedora Linux
> distributions that don't a) support Flatpak and b) have an appropriate
> Flatpak runtime for running FMW on (beyond compiling it yourself, I
> guess).

I don't understand this. How does blocking Fedora help if some
non-Fedora distro can't write an otherwise valid Fedora image?

The ISO creation method isn't changing, and FMW effectively does a
write that's dd-based, therefore if FMW works on Fedora, dd would work
also so long as the distro's copy of dd works.



> If we do not 'support' livecd-iso-to-disk any more, we no longer
> support:
>
> 1) persistent storage (via overlays)
> 2) non-destructive write

Is someone able to take over for Brian (bcl)  as the maintainer, and
will be able to provide quick fixes that would otherwise block the
release? Until that's established, then I'd say it has insufficient
support for its bugs to be release blocking.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Neal Gompa]

On Mon, Oct 3, 2016 at 3:07 PM, Adam Williamson
 wrote:
>
> If we do not 'support' livecd-iso-to-disk any more, we no longer
> support:
>
> 1) persistent storage (via overlays)
> 2) non-destructive write

Does anyone know why we can't have Fedora Media Writer support these
functions as well? I would suggest that making Fedora Media Writer the
only supported method means that it needs to be able to support all
the ways Fedora can be used on a USB stick. Until it can, it can't be
the sole mechanism.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Adam Williamson]

On Mon, 2016-10-03 at 12:49 -0600, Chris Murphy wrote:
> Based on today's blocker review meeting discussion, and this email
> thread [1] I'd like to propose making only Fedora Media Writer the
> *officially supported* USB installation media creation tool, starting
> with Fedora 26.
> 
> The practical implication of "officially support" means bugs for which
> we'd block the release. It doesn't make sense to block the release if
> myriad tools all don't succeed. We only really need one to work, and
> Fedora Media Writer is the cross platform tool we're investing in long
> term.
> 
> The main idea of the proposal is to no longer block the release when
> Fedora Media Writer is working, but some other possibly useful ways of
> creating media aren't working. It doesn't mean those tools won't be
> fixed, or would be removed from the distribution, just means we're not
> holding up release for those alternative tools.
> 
> Comments?

There is currently no real way to use FMW on non-Fedora Linux
distributions that don't a) support Flatpak and b) have an appropriate
Flatpak runtime for running FMW on (beyond compiling it yourself, I
guess).

If we do not 'support' livecd-iso-to-disk any more, we no longer
support:

1) persistent storage (via overlays)
2) non-destructive write
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

F26 proposal: Make Fedora Media Writer the officially supported USB install media creator

[Chris Murphy]

Based on today's blocker review meeting discussion, and this email
thread [1] I'd like to propose making only Fedora Media Writer the
*officially supported* USB installation media creation tool, starting
with Fedora 26.

The practical implication of "officially support" means bugs for which
we'd block the release. It doesn't make sense to block the release if
myriad tools all don't succeed. We only really need one to work, and
Fedora Media Writer is the cross platform tool we're investing in long
term.

The main idea of the proposal is to no longer block the release when
Fedora Media Writer is working, but some other possibly useful ways of
creating media aren't working. It doesn't mean those tools won't be
fixed, or would be removed from the distribution, just means we're not
holding up release for those alternative tools.

Comments?



[1]

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/MVXA4CAEYPNX2TIZCETQM52EQCYMWXU5/



-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org