Re: [OE-core] [oe] RFC: Reference updater filesystem

[Jens Rehsack]

> Am 03.12.2015 um 17:45 schrieb Mariano Lopez :
> 
> [...]
> 
> Thanks to all for your input. The conclusion of this thread is:
> 
> 1. One size doesn't fit all.
> 2. Most of the people was fine with the image based update.

Where is the image stored? I read over referred 
http://sbabic.github.io/swupdate/
and realized that the update image must include a stripped rootfs and the entire
software for rootfs as well as probably the recoveryfs/maintfs and some logic
for updating the data volume (eg. migrator for database of business logic).

This can be very large ... and must be stored somewhere.

Next question: how bullet proof is the swupdate?

Out process is designed (because of typical users can't interact) to can survive
surprising power outage. Unfortunately we didn't manage to split out u-boot into
separate updatable parts, so we can't benefit from it - but technically (if one
has implemented for heir targets), our script updates and continues after power
comes back.

I think, beside the question of having an image containing the update (we have 
such
an add-on, too - for legacy devices without he modern Yocto based firmware), the
question of stability should be evaluated, too.

> 3. The recommended way to keep the configuration is to have a separated data 
> partition.
> 4. The partition scheme would be:
>1. boot. This is the usual boot partition
>2. rootfs. Partition used for normal operation.
>3. maintenance. This partition will be used to update rootfs.
>4. data. This will hold the configuration files. Not modified by
>updates.

Cheers
-- 
Jens Rehsack - rehs...@gmail.com

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe] RFC: Reference updater filesystem

[Mariano Lopez]

On 11/24/2015 12:47 PM, Roman Khimov wrote:

В письме от 24 ноября 2015 12:27:30 пользователь Mark Hatle написал:

On 11/24/15 12:05 PM, Roman Khimov wrote:

В письме от 24 ноября 2015 07:47:38 пользователь Mark Hatle написал:

On 11/24/15 4:39 AM, Roman Khimov wrote:

В письме от 23 ноября 2015 15:41:28 пользователь Mariano Lopez написал:

1. Use a separate partition for the configuration.

a. The pro of this method is the partition is not touched during the

update.

b. The con of this method is the configuration is not directly in

rootfs (example: /etc).

That's the right solution, although to do it really right (at least IMO)
you need to implement the /usr merge [1] (and that's orthogonal to using
or not using systemd), which can also help you make your /usr read-only
(because that's just code and static data) with read-write / for user
data of various sorts.

Why does merging /usr have anything to do with this?  I've read the case
for merging /usr and / and still don't understand why it "helps".  The
key is that if you have separate partitions for /usr and /, then you
need to update both of them in sequence.  Merging these two just seems
like a lazy solution to people not wanting to deal with early boot being
self-contained.

It helps in that you can achieve a clear separation of your software and
users data (whatever that is, even if that's just some configuration
files) and only update your part (which is /usr).

This can easily be archived using different partitions like /opt as well.

Usually it leads to more complicated partitioning and more complicated startup
(as well as update) procedure. Of course it can vary from system to system,
but in my experience /usr merge was exactly the thing needed to clearly
separate and simplify things. One way or another you want to have your code
and users data to be separated, /usr merge makes it quite simple.


Bandwidth is a big deal in areas that are not serviced by anything but very
GPRS or worse.  So 100 MB is even too big in some cases.  Cost is a factor
as is overall bandwidth.

Again, it depends on the product and what you are building if this matters.

The point is we don't want to make some assumptions that preclude
alternative implementations.

Sure.



Thanks to all for your input. The conclusion of this thread is:

1. One size doesn't fit all.
2. Most of the people was fine with the image based update.
3. The recommended way to keep the configuration is to have a separated 
data partition.

4. The partition scheme would be:
1. boot. This is the usual boot partition
2. rootfs. Partition used for normal operation.
3. maintenance. This partition will be used to update rootfs.
4. data. This will hold the configuration files. Not modified by
updates.


--
Mariano Lopez
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe] RFC: Reference updater filesystem

[Roman Khimov]

В письме от 24 ноября 2015 10:30:17 пользователь Lopez, Mariano написал:
> > That's the right solution, although to do it really right (at least IMO)
> > you need to implement the /usr merge [1] (and that's orthogonal to using
> > or not using systemd), which can also help you make your /usr read-only
> > (because that's just code and static data) with read-write / for user
> > data of various sorts.
> 
> To be honest I'm not familiar with /usr merge, I need to check on that
> to see if it is a good option with the current OE-core infrastructure.

It needs some patches for current OE, I hope I'll find some time to rebase the 
set I have for this (based on fido) and send it for discussion.

-- 
 http://roman.khimov.ru
mailto: ro...@khimov.ru
gpg --keyserver hkp://subkeys.pgp.net --recv-keys 0xE5E055C3
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe] RFC: Reference updater filesystem

[Mark Hatle]

On 11/24/15 12:05 PM, Roman Khimov wrote:
> В письме от 24 ноября 2015 07:47:38 пользователь Mark Hatle написал:
>> On 11/24/15 4:39 AM, Roman Khimov wrote:
>>> В письме от 23 ноября 2015 15:41:28 пользователь Mariano Lopez написал:
 1. Use a separate partition for the configuration.

a. The pro of this method is the partition is not touched during the

 update.

b. The con of this method is the configuration is not directly in

 rootfs (example: /etc).
>>>
>>> That's the right solution, although to do it really right (at least IMO)
>>> you need to implement the /usr merge [1] (and that's orthogonal to using
>>> or not using systemd), which can also help you make your /usr read-only
>>> (because that's just code and static data) with read-write / for user
>>> data of various sorts.
>>
>> Why does merging /usr have anything to do with this?  I've read the case for
>> merging /usr and / and still don't understand why it "helps".  The key is
>> that if you have separate partitions for /usr and /, then you need to
>> update both of them in sequence.  Merging these two just seems like a lazy
>> solution to people not wanting to deal with early boot being
>> self-contained.
> 
> It helps in that you can achieve a clear separation of your software and 
> users 
> data (whatever that is, even if that's just some configuration files) and 
> only 
> update your part (which is /usr).

This can easily be archived using different partitions like /opt as well.

>> So going back to image upgrade.  The key here is that you need a way to
>> update arbitrary images with arbitrary contents and a mechanisms that is
>> smart enough to generate the update (vs a full image flash) to conserve
>> bandwidth.
> 
> In my experience, size is almost not an issue these days, at least if we're 
> talking about something less than 100 MB, but updating the whole image is 
> more 
> reliable and easier to manage.
> 

Bandwidth is a big deal in areas that are not serviced by anything but very GPRS
or worse.  So 100 MB is even too big in some cases.  Cost is a factor as is
overall bandwidth.

Again, it depends on the product and what you are building if this matters.

The point is we don't want to make some assumptions that preclude alternative
implementations.

--Mark
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe] RFC: Reference updater filesystem

[Roman Khimov]

В письме от 24 ноября 2015 12:27:30 пользователь Mark Hatle написал:
> On 11/24/15 12:05 PM, Roman Khimov wrote:
> > В письме от 24 ноября 2015 07:47:38 пользователь Mark Hatle написал:
> >> On 11/24/15 4:39 AM, Roman Khimov wrote:
> >>> В письме от 23 ноября 2015 15:41:28 пользователь Mariano Lopez написал:
>  1. Use a separate partition for the configuration.
>  
> a. The pro of this method is the partition is not touched during the
>  
>  update.
>  
> b. The con of this method is the configuration is not directly in
>  
>  rootfs (example: /etc).
> >>> 
> >>> That's the right solution, although to do it really right (at least IMO)
> >>> you need to implement the /usr merge [1] (and that's orthogonal to using
> >>> or not using systemd), which can also help you make your /usr read-only
> >>> (because that's just code and static data) with read-write / for user
> >>> data of various sorts.
> >> 
> >> Why does merging /usr have anything to do with this?  I've read the case
> >> for merging /usr and / and still don't understand why it "helps".  The
> >> key is that if you have separate partitions for /usr and /, then you
> >> need to update both of them in sequence.  Merging these two just seems
> >> like a lazy solution to people not wanting to deal with early boot being
> >> self-contained.
> > 
> > It helps in that you can achieve a clear separation of your software and
> > users data (whatever that is, even if that's just some configuration
> > files) and only update your part (which is /usr).
> 
> This can easily be archived using different partitions like /opt as well.

Usually it leads to more complicated partitioning and more complicated startup 
(as well as update) procedure. Of course it can vary from system to system, 
but in my experience /usr merge was exactly the thing needed to clearly 
separate and simplify things. One way or another you want to have your code 
and users data to be separated, /usr merge makes it quite simple.

> Bandwidth is a big deal in areas that are not serviced by anything but very
> GPRS or worse.  So 100 MB is even too big in some cases.  Cost is a factor
> as is overall bandwidth.
> 
> Again, it depends on the product and what you are building if this matters.
> 
> The point is we don't want to make some assumptions that preclude
> alternative implementations.

Sure.

-- 
 http://roman.khimov.ru
mailto: ro...@khimov.ru
gpg --keyserver hkp://subkeys.pgp.net --recv-keys 0xE5E055C3
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe] RFC: Reference updater filesystem

[Lopez, Mariano]

On 11/24/2015 4:30 AM, Roman Khimov wrote:

В письме от 23 ноября 2015 15:41:28 пользователь Mariano Lopez написал:

1. Use a separate partition for the configuration.
a. The pro of this method is the partition is not touched during the
update.
b. The con of this method is the configuration is not directly in
rootfs (example: /etc).

That's the right solution, although to do it really right (at least IMO) you
need to implement the /usr merge [1] (and that's orthogonal to using or not
using systemd), which can also help you make your /usr read-only (because
that's just code and static data) with read-write / for user data of various
sorts.


To be honest I'm not familiar with /usr merge, I need to check on that 
to see if it is a good option with the current OE-core infrastructure.





3. Have an OverlayFS for the rootfs or the partition that have the
configuration.
a. The pro is the configuration is  "directly" in rootfs.
b. The con is there is need to provide a custom init to guarantee the
Overlay is mounted before the boot process.

And this is the approach I would recommend not doing. I've used UnionFS for
thing like that (overlaying whole root file system) some 6 years ago, it
sounded nice and it kinda worked, but it wasn't difficult to make it fail
(just a little playing with power), we've even seen failures on production
devices, like when you have whiteout file for directory already written, but
don't have new files in it yet and that can completely ruin the system.

Also, it usually works better when you don't have any changes in the lower
layer, but we're talking about updating it here, you can easily end up in a
situation where you have updated something in the rootfs but that was
overriden by upper layer and thus your user doesn't see any change.


Thanks for sharing your experience, this is another big con for the 
Overlay option.





With the above information I'm proposing to use a separate partition for
the configuration; this is because is more reliable and doesn't require
big changes in the current architecture.

So, the idea is to have 4 partitions in the media:
1. boot. This is the usual boot partition
2. data. This will hold the configuration files. Not modified by updates.
3. maintenance. This partition will be used to update rootfs.
4. rootfs. Partition used for normal operation.

You probably don't need to separate 1 and 3, all the code for system update
should easily fit into initramfs and just making /boot a bit larger would
allow you to store some backup rootfs.


I left the /boot partition separate just in case there is need to 
replace the kernel or the bootloader. This way it would be easier to 
change using the same method as the upgrading the rootfs.




Also, you can swap 4 and 2 which will be useful if you're installing on
different sized storage devices, usually you know good enough the size of your
rootfs, but you probably want to leave more space for user data if there is an
opportunity to do so, that's just easier to do with data partition at the end.


I was thinking in the same thinking just backwards, usually 
configuration files are just small text files that don't require too 
much space. If you require a new feature in the target that will make 
rootfs to grow depending on the feature. I plan to use wic to accomplish 
the filesystem structure. A good thing about wic is that it will be very 
easy to do the swap, just need to modify two options in the .wks file.





[1] http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe] RFC: Reference updater filesystem

[Roman Khimov]

В письме от 24 ноября 2015 07:47:38 пользователь Mark Hatle написал:
> On 11/24/15 4:39 AM, Roman Khimov wrote:
> > В письме от 23 ноября 2015 15:41:28 пользователь Mariano Lopez написал:
> >> 1. Use a separate partition for the configuration.
> >> 
> >>a. The pro of this method is the partition is not touched during the
> >> 
> >> update.
> >> 
> >>b. The con of this method is the configuration is not directly in
> >> 
> >> rootfs (example: /etc).
> > 
> > That's the right solution, although to do it really right (at least IMO)
> > you need to implement the /usr merge [1] (and that's orthogonal to using
> > or not using systemd), which can also help you make your /usr read-only
> > (because that's just code and static data) with read-write / for user
> > data of various sorts.
> 
> Why does merging /usr have anything to do with this?  I've read the case for
> merging /usr and / and still don't understand why it "helps".  The key is
> that if you have separate partitions for /usr and /, then you need to
> update both of them in sequence.  Merging these two just seems like a lazy
> solution to people not wanting to deal with early boot being
> self-contained.

It helps in that you can achieve a clear separation of your software and users 
data (whatever that is, even if that's just some configuration files) and only 
update your part (which is /usr).

> So going back to image upgrade.  The key here is that you need a way to
> update arbitrary images with arbitrary contents and a mechanisms that is
> smart enough to generate the update (vs a full image flash) to conserve
> bandwidth.

In my experience, size is almost not an issue these days, at least if we're 
talking about something less than 100 MB, but updating the whole image is more 
reliable and easier to manage.

-- 
 http://roman.khimov.ru
mailto: ro...@khimov.ru
gpg --keyserver hkp://subkeys.pgp.net --recv-keys 0xE5E055C3
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe] RFC: Reference updater filesystem

[Roman Khimov]

В письме от 23 ноября 2015 15:41:28 пользователь Mariano Lopez написал:
> 1. Use a separate partition for the configuration.
>a. The pro of this method is the partition is not touched during the
> update.
>b. The con of this method is the configuration is not directly in
> rootfs (example: /etc).

That's the right solution, although to do it really right (at least IMO) you 
need to implement the /usr merge [1] (and that's orthogonal to using or not 
using systemd), which can also help you make your /usr read-only (because 
that's just code and static data) with read-write / for user data of various 
sorts.

> 3. Have an OverlayFS for the rootfs or the partition that have the
> configuration.
>a. The pro is the configuration is  "directly" in rootfs.
>b. The con is there is need to provide a custom init to guarantee the
> Overlay is mounted before the boot process.

And this is the approach I would recommend not doing. I've used UnionFS for 
thing like that (overlaying whole root file system) some 6 years ago, it 
sounded nice and it kinda worked, but it wasn't difficult to make it fail 
(just a little playing with power), we've even seen failures on production 
devices, like when you have whiteout file for directory already written, but 
don't have new files in it yet and that can completely ruin the system.

Also, it usually works better when you don't have any changes in the lower 
layer, but we're talking about updating it here, you can easily end up in a 
situation where you have updated something in the rootfs but that was 
overriden by upper layer and thus your user doesn't see any change.

> With the above information I'm proposing to use a separate partition for
> the configuration; this is because is more reliable and doesn't require
> big changes in the current architecture.
> 
> So, the idea is to have 4 partitions in the media:
> 1. boot. This is the usual boot partition
> 2. data. This will hold the configuration files. Not modified by updates.
> 3. maintenance. This partition will be used to update rootfs.
> 4. rootfs. Partition used for normal operation.

You probably don't need to separate 1 and 3, all the code for system update 
should easily fit into initramfs and just making /boot a bit larger would 
allow you to store some backup rootfs.

Also, you can swap 4 and 2 which will be useful if you're installing on 
different sized storage devices, usually you know good enough the size of your 
rootfs, but you probably want to leave more space for user data if there is an 
opportunity to do so, that's just easier to do with data partition at the end.


[1] http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core