Re: [OE-core] RFC: Reference updater filesystem

2015-11-24 Thread Lopez, Mariano 

On 11/24/2015 1:32 AM, Randy Witt wrote:



On Mon, Nov 23, 2015 at 1:41 PM, Mariano Lopez 
> 
wrote:


There has been interest in an image based software updater in
Yocto Project. The proposed solution for a image based updater is
to use Stefano Babic's software updater
(http://sbabic.github.io/swupdate). This software do a binary
copy, so it is needed to have at least two partitions, these
partitions would be the rootfs and the maintenance partition. The
rootfs it's the main partition used to boot during the normal
device operation, on the other hand, the maintenance will be used
to update the main partition.

To update the system, the user has to connect to device and boot
in the maintenance partition; once in the maintenance partition
the software updater will copy the new image in the rootfs
partition. A final reboot into the rootfs it is necessary to
complete the upgrade.

As mentioned before the the software will copy an image to the
partition, so everything in that partition will be wiped out,
including custom configurations. To avoid the loss of
configuration I explore three different solutions:
1. Use a separate partition for the configuration.
  a. The pro of this method is the partition is not touched during
the update.
  b. The con of this method is the configuration is not directly
in rootfs (example: /etc).

Configuration files can be anywhere a package decides to install them. 
So having a single partition would be difficult. If you could, you 
would most likely be forced to have an initramfs to make sure /etc was 
mounted before init runs.


/etc was an example, the image should have the required files to make 
the target boot and the get the application configuration from this 
other partition. This is like openwrt does, it has a read-only rootfs 
and small read-write partition where the user can write its 
configuration and restore it at boot time.



2. Do the backup during the update.
  a. The pro is the configuration is directly in rootfs.
  b. The con is If the update fail most likely the configuration
would be lost.

Why would the configuration be lost if the update fails? Couldn't it 
just be stored on the thumbdrive?


If there is a power loss while the configuration is copied, the 
partition could go corrupt and would be difficult to recover. And as you 
mentioned  before the configuration files could be anywhere, so the 
script must be customized to get all those files and once the update is 
complete another script must restore those files, these could be 
cumbersome instead of the application have the config in another partition.



3. Have an OverlayFS for the rootfs or the partition that have the
configuration.
  a. The pro is the configuration is  "directly" in rootfs.
  b. The con is there is need to provide a custom init to
guarantee the Overlay is mounted before the boot process.

With the above information I'm proposing to use a separate
partition for the configuration; this is because is more reliable
and doesn't require big changes in the current architecture.

So, the idea is to have 4 partitions in the media:
1. boot. This is the usual boot partition
2. data. This will hold the configuration files. Not modified by
updates.
3. maintenance. This partition will be used to update rootfs.
4. rootfs. Partition used for normal operation.

Mariano
-- 
___

Openembedded-core mailing list
Openembedded-core@lists.openembedded.org

http://lists.openembedded.org/mailman/listinfo/openembedded-core




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] RFC: Reference updater filesystem

2015-11-23 Thread Anders Darander 
* Mariano Lopez  [151123 22:41]:

> There has been interest in an image based software updater in Yocto Project.

Ok. Sure, it might be nice with something that can be shared, instead
off everyone's building our own solutions.

> The proposed solution for a image based updater is to use Stefano Babic's
> software updater (http://sbabic.github.io/swupdate). This software do a
> binary copy, so it is needed to have at least two partitions, these
> partitions would be the rootfs and the maintenance partition. The rootfs
> it's the main partition used to boot during the normal device operation, on
> the other hand, the maintenance will be used to update the main partition.

I haven't checked the swupdate tool, though I'd suspect that it also
supports the alternating rootfs use case? (I.e. run system1 update
system2; reboot to system2. Next update is system1). This is a rather
common setup, not least when you need a remote upgrade facility.

Would your proposed inclusion to the Yocto Project support that case
too?

> To update the system, the user has to connect to device and boot in the
> maintenance partition; once in the maintenance partition the software
> updater will copy the new image in the rootfs partition. A final reboot into
> the rootfs it is necessary to complete the upgrade.

Like said above, not all system can be reached manually (at least not in
cost efficient way). Sure, the mainenance partition scheme can be made
to work anyway...

> As mentioned before the the software will copy an image to the partition, so
> everything in that partition will be wiped out, including custom
> configurations. To avoid the loss of configuration I explore three different
> solutions:
> 1. Use a separate partition for the configuration.
>   a. The pro of this method is the partition is not touched during the
> update.
>   b. The con of this method is the configuration is not directly in rootfs
> (example: /etc).

I'd vote for that as well. Though, I only keep the re-writable
configurations here. The one that are constant between all systems are
shipped in /etc in the read-only-rootfs.

> With the above information I'm proposing to use a separate partition for the
> configuration; this is because is more reliable and doesn't require big
> changes in the current architecture.

> So, the idea is to have 4 partitions in the media:
> 1. boot. This is the usual boot partition
> 2. data. This will hold the configuration files. Not modified by updates.
> 3. maintenance. This partition will be used to update rootfs.
> 4. rootfs. Partition used for normal operation.

How flexible to you intend to make this system? Allow everything that
swupdate supports? Or a specific subset?

Cheers,
Anders

-- 
Anders Darander, Senior System Architect
ChargeStorm AB / eStorm AB
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] RFC: Reference updater filesystem

2015-11-23 Thread Randy Witt 
On Mon, Nov 23, 2015 at 1:41 PM, Mariano Lopez <
mariano.lo...@linux.intel.com> wrote:

> There has been interest in an image based software updater in Yocto
> Project. The proposed solution for a image based updater is to use Stefano
> Babic's software updater (http://sbabic.github.io/swupdate). This
> software do a binary copy, so it is needed to have at least two partitions,
> these partitions would be the rootfs and the maintenance partition. The
> rootfs it's the main partition used to boot during the normal device
> operation, on the other hand, the maintenance will be used to update the
> main partition.
>
> To update the system, the user has to connect to device and boot in the
> maintenance partition; once in the maintenance partition the software
> updater will copy the new image in the rootfs partition. A final reboot
> into the rootfs it is necessary to complete the upgrade.
>
> As mentioned before the the software will copy an image to the partition,
> so everything in that partition will be wiped out, including custom
> configurations. To avoid the loss of configuration I explore three
> different solutions:
> 1. Use a separate partition for the configuration.
>   a. The pro of this method is the partition is not touched during the
> update.
>   b. The con of this method is the configuration is not directly in rootfs
> (example: /etc).
>
> Configuration files can be anywhere a package decides to install them. So
having a single partition would be difficult. If you could, you would most
likely be forced to have an initramfs to make sure /etc was mounted before
init runs.


> 2. Do the backup during the update.
>   a. The pro is the configuration is directly in rootfs.
>   b. The con is If the update fail most likely the configuration would be
> lost.
>
> Why would the configuration be lost if the update fails? Couldn't it just
be stored on the thumbdrive?


> 3. Have an OverlayFS for the rootfs or the partition that have the
> configuration.
>   a. The pro is the configuration is  "directly" in rootfs.
>   b. The con is there is need to provide a custom init to guarantee the
> Overlay is mounted before the boot process.
>
> With the above information I'm proposing to use a separate partition for
> the configuration; this is because is more reliable and doesn't require big
> changes in the current architecture.
>
> So, the idea is to have 4 partitions in the media:
> 1. boot. This is the usual boot partition
> 2. data. This will hold the configuration files. Not modified by updates.
> 3. maintenance. This partition will be used to update rootfs.
> 4. rootfs. Partition used for normal operation.
>
> Mariano
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] RFC: Reference updater filesystem

2015-11-23 Thread Mariano Lopez 
There has been interest in an image based software updater in Yocto 
Project. The proposed solution for a image based updater is to use 
Stefano Babic's software updater (http://sbabic.github.io/swupdate). 
This software do a binary copy, so it is needed to have at least two 
partitions, these partitions would be the rootfs and the maintenance 
partition. The rootfs it's the main partition used to boot during the 
normal device operation, on the other hand, the maintenance will be used 
to update the main partition.


To update the system, the user has to connect to device and boot in the 
maintenance partition; once in the maintenance partition the software 
updater will copy the new image in the rootfs partition. A final reboot 
into the rootfs it is necessary to complete the upgrade.


As mentioned before the the software will copy an image to the 
partition, so everything in that partition will be wiped out, including 
custom configurations. To avoid the loss of configuration I explore 
three different solutions:

1. Use a separate partition for the configuration.
  a. The pro of this method is the partition is not touched during the 
update.
  b. The con of this method is the configuration is not directly in 
rootfs (example: /etc).


2. Do the backup during the update.
  a. The pro is the configuration is directly in rootfs.
  b. The con is If the update fail most likely the configuration would 
be lost.


3. Have an OverlayFS for the rootfs or the partition that have the 
configuration.

  a. The pro is the configuration is  "directly" in rootfs.
  b. The con is there is need to provide a custom init to guarantee the 
Overlay is mounted before the boot process.


With the above information I'm proposing to use a separate partition for 
the configuration; this is because is more reliable and doesn't require 
big changes in the current architecture.


So, the idea is to have 4 partitions in the media:
1. boot. This is the usual boot partition
2. data. This will hold the configuration files. Not modified by updates.
3. maintenance. This partition will be used to update rootfs.
4. rootfs. Partition used for normal operation.

Mariano
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core