Hello community, here is the log from the commit of package kiwi for openSUSE:Factory checked in at 2016-01-28 17:18:30 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/kiwi (Old) and /work/SRC/openSUSE:Factory/.kiwi.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "kiwi" Changes: -------- --- /work/SRC/openSUSE:Factory/kiwi/kiwi.changes 2016-01-05 09:40:35.000000000 +0100 +++ /work/SRC/openSUSE:Factory/.kiwi.new/kiwi.changes 2016-01-28 17:18:34.000000000 +0100 @@ -1,0 +2,146 @@ +Thu Jan 14 12:53:29 CET 2016 - [email protected] + +- v7.03.49 released + +------------------------------------------------------------------- +Tue Jan 12 15:39:39 CET 2016 - [email protected] + +- Don't use filesystem specific mount option + + When mounting the read write filesystem, prevent using + filesystem specific options like barrier + +------------------------------------------------------------------- +Tue Jan 12 15:38:08 CET 2016 - [email protected] + +- Use HYBRID_EXT4_OPTS in fat container filesystem + +------------------------------------------------------------------- +Tue Jan 12 15:34:40 CET 2016 - [email protected] + +- Check read write filesystem prior to mounting + + If a filesystem type could be identified on the read write + partition check it before mounting to eliminate a potential + dirty state + +------------------------------------------------------------------- +Tue Jan 12 15:26:55 CET 2016 - [email protected] + +- Cascade exfat mount + + try normal mount first, if this does not work try fuse mount + Normally this is done automatically by the mount program but + if not we will give the direct fuse mount a chance + +------------------------------------------------------------------- +Mon Jan 11 19:11:12 CET 2016 - [email protected] + +- Make createFileSystem more robust + + When passing in a loop file instead of a device name, the method + should be smart enough to handle the name without shell evaluation + +------------------------------------------------------------------- +Mon Jan 11 16:38:32 CET 2016 - [email protected] + +- Compile rnc -> rng + +------------------------------------------------------------------- +Mon Jan 11 16:30:09 CET 2016 - [email protected] + +- Added support for exfat as hybrid filesystem + + In addition to fat also exfat is now supported as persistent + write filesystem. Because of the limitations of a fat filesystem + fat and exfat are only used as a container filesystem providing + an ext4 linux filesystem as a cowfile. The algorithm to create + the size of the cow file has also been changed to use half of + the size of the write partition or on fat a max size of 4G. + The size of the cowfile is also prepared to become overwritten. + However the XML definition and implementation to do this + is still missing + +------------------------------------------------------------------- +Mon Jan 11 16:08:17 CET 2016 - [email protected] + +- Reread partition table after hybrid setup + + The hybrid write partition is created via fdisk. Some version + of fdisk does not send the ioctl to let the kernel reread the + table or fdisk itself holds it busy. Thus we actively initiate + a reread via blockdev + +------------------------------------------------------------------- +Mon Jan 11 16:05:28 CET 2016 - [email protected] + +- Optimize ext4 hybrid write filesystem options + + Optimized for 512kB erase block size + +------------------------------------------------------------------- +Mon Jan 11 12:54:57 CET 2016 - [email protected] + +- Use mount options to increase overlay performace + + For overlay filesystems not writing into a tmpfs performance is + more important than safety. We use this combination of options + for now, if you encounter stability problems please let us know + +------------------------------------------------------------------- +Mon Jan 11 12:32:43 CET 2016 - [email protected] + +- Use -f force option for extX filesystem checker + +------------------------------------------------------------------- +Mon Jan 11 12:25:15 CET 2016 - [email protected] + +- Refactor and cleanup setupReadWrite + + Fix misleading error message and refactor the code to be less + complex and more clear in the processing of tasks + +------------------------------------------------------------------- +Mon Jan 11 11:38:59 CET 2016 - [email protected] + +- Allow custom cowfile name for persistent data + + Instead of a fixed name 'cowfile' we allow a custom name which is + predefined in HYBRID_PERSISTENT_FILENAME and prepared to become + overwritten by an XML defintion whose implementation will follow + later. Reason for the change is that a cowfile is visible as + plain data file to the operating system if e.g used on a live + stick. It should be more clear to the user what this file is + good for + +------------------------------------------------------------------- +Mon Jan 11 11:29:23 CET 2016 - [email protected] + +- Protect mkfs.exfat from being deleted + + Added to the strip tools section in order to keep it in the initrd + +------------------------------------------------------------------- +Mon Jan 11 11:26:23 CET 2016 - [email protected] + +- Added support for exfat creation and probing + + In preparation to use exfat instead of vfat in a loop container + for persistent data it's required to provide support for mkfs.exfat + +------------------------------------------------------------------- +Mon Jan 11 11:20:24 CET 2016 - [email protected] + +- Make loop_setup and loop_delete more robust + + The methods did not cope well with filenames containing bash + characters with special meaning e.g spaces. For use with a + iso hybrid cowfile whose name is visible in the OS the methods + should be able to work with any given filename + +------------------------------------------------------------------- +Sat Dec 26 13:55:55 CET 2015 - [email protected] + +- fix message of missing command + +------------------------------------------------------------------- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ kiwi.spec ++++++ --- /var/tmp/diff_new_pack.oBtAOh/_old 2016-01-28 17:18:35.000000000 +0100 +++ /var/tmp/diff_new_pack.oBtAOh/_new 2016-01-28 17:18:35.000000000 +0100 @@ -27,7 +27,7 @@ Name: kiwi License: GPL-2.0 Group: System/Management -Version: 7.03.48 +Version: 7.03.49 Provides: kiwi-schema = 6.2 Provides: kiwi-image:aci Provides: kiwi-image:lxc ++++++ kiwi-docu.tar.bz2 ++++++ ++++++ kiwi-repo.tar.bz2 ++++++ ++++++ kiwi.tar.bz2 ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/kiwi/.revision new/kiwi/.revision --- old/kiwi/.revision 2015-12-17 09:28:28.000000000 +0100 +++ new/kiwi/.revision 2015-12-17 09:28:28.000000000 +0100 @@ -1 +1 @@ -e7ef2da84a78034424ae14ba4e6b2ebb14592dff +93052054fee85c16a6b1236a5d23144c77542f3a diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/kiwi/metadata/KIWIConfig.xml new/kiwi/metadata/KIWIConfig.xml --- old/kiwi/metadata/KIWIConfig.xml 2015-09-21 16:23:18.000000000 +0200 +++ new/kiwi/metadata/KIWIConfig.xml 2016-01-14 12:53:04.000000000 +0100 @@ -164,6 +164,7 @@ <file name="mkfs.ext4"/> <file name="mkfs.ntfs"/> <file name="mkfs.xfs"/> + <file name="mkfs.exfat"/> <file name="xfs_growfs"/> <file name="xfs_repair"/> <file name="xfs_admin"/> @@ -240,6 +241,7 @@ <file name="stat"/> <file name="swapoff"/> <file name="swapon"/> + <file name="systemd-detect-virt"/> <file name="tac"/> <file name="rev"/> <file name="tail"/> diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/kiwi/modules/KIWIGlobals.pm new/kiwi/modules/KIWIGlobals.pm --- old/kiwi/modules/KIWIGlobals.pm 2015-12-23 10:50:16.000000000 +0100 +++ new/kiwi/modules/KIWIGlobals.pm 2016-01-14 12:53:24.000000000 +0100 @@ -1933,7 +1933,7 @@ # Globals (generic) #------------------------------------------ my %data; - $data{Version} = "7.03.48"; + $data{Version} = "7.03.49"; $data{Publisher} = "SUSE LINUX GmbH"; $data{Preparer} = "KIWI - http://opensuse.github.com/kiwi";; $data{ConfigName} = "config.xml"; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/kiwi/modules/KIWILinuxRC.sh new/kiwi/modules/KIWILinuxRC.sh --- old/kiwi/modules/KIWILinuxRC.sh 2015-12-23 10:50:16.000000000 +0100 +++ new/kiwi/modules/KIWILinuxRC.sh 2016-01-14 12:50:28.000000000 +0100 @@ -23,6 +23,7 @@ export TRANSFER_ERRORS_FILE=/tmp/transfer.errors export UFONT=/usr/share/fbiterm/fonts/b16.pcf.gz export CONSOLE_FONT=/usr/share/kbd/consolefonts/default8x16.gz +export HYBRID_PERSISTENT_FILENAME="Live OS's persistent storage.fs" export HYBRID_PERSISTENT_FS=btrfs export HYBRID_PERSISTENT_ID=83 export HYBRID_PERSISTENT_DIR=/read-write @@ -41,6 +42,12 @@ } #====================================== +# Exports (hybrid filesystem options) +#-------------------------------------- +# Optimized for 512kB erase block size +export HYBRID_EXT4_OPTS="-b 4096 -O ^has_journal -E stride=128,stripe-width=128" + +#====================================== # Exports (console) #-------------------------------------- test -z "$ELOG_BOOTSHELL" && export ELOG_BOOTSHELL=/dev/tty2 @@ -3930,6 +3937,7 @@ xfs) FSTYPE=xfs ;; udf) FSTYPE=udf ;; zfs_member) FSTYPE=zfs ;; + exfat) FSTYPE=exfat ;; *) FSTYPE=unknown ;; @@ -4141,7 +4149,7 @@ #====================================== # Manual loading of modules #-------------------------------------- - for i in rd brd edd dm-mod xennet xenblk virtio_blk loop squashfs;do + for i in rd brd edd dm-mod xennet xenblk virtio_blk loop squashfs fuse;do modprobe $i &>/dev/null done udevPending @@ -6420,32 +6428,41 @@ # ---- local IFS=$IFS_ORIG local rwDir=/read-write - local rwDevice=`echo $UNIONFS_CONFIG | cut -d , -f 1` + local rwDevice=$(echo $UNIONFS_CONFIG | cut -d , -f 1) + local fstype=$(blkid $rwDevice -s TYPE -o value) + local hybrid_fs=$HYBRID_PERSISTENT_FS + local create_hybrid="no" + local fs_opts mkdir -p $rwDir - if [ $LOCAL_BOOT = "no" ] && [ $systemIntegrity = "clean" ];then - if [ "$RELOAD_IMAGE" = "yes" ] || \ - ! mount -o ro $rwDevice $rwDir &>/dev/null + if [ $hybrid_fs = "ext4" ];then + fs_opts="$HYBRID_EXT4_OPTS" + fi + if [ ! -z "$kiwi_hybridpersistent_filesystem" ];then + hybrid_fs=$kiwi_hybridpersistent_filesystem + fi + if [ $LOCAL_BOOT = "yes" ] || [ ! $systemIntegrity = "clean" ];then + # no further action on a standard boot/reboot or in unclean state + return 0 + fi + if [ ! -z "$fstype" ];then + Echo "Checking filesystem for RW data on $rwDevice..." + checkFilesystem $rwDevice &>/dev/null + fi + if [ "$RELOAD_IMAGE" = "yes" ]; then + # create rw fs explicitly activated + create_hybrid="yes" + elif ! mount -o ro $rwDevice $rwDir &>/dev/null; then + # still failing after check, trigger creation of new rw fs + create_hybrid="yes" + fi + mountpoint -q $rwDir && umount $rwDevice + if [ "$create_hybrid" = "yes" ];then + Echo "Creating filesystem for RW data on $rwDevice..." + if ! createFilesystem \ + "$rwDevice" "" "" "hybrid" "false" "$hybrid_fs" "$fs_opts" then - local hybrid_fs=$HYBRID_PERSISTENT_FS - if [ ! -z "$kiwi_hybridpersistent_filesystem" ];then - hybrid_fs=$kiwi_hybridpersistent_filesystem - fi - Echo "Checking filesystem for RW data on $rwDevice..." - checkFilesystem $rwDevice - - if [ "$RELOAD_IMAGE" = "yes" ] || \ - ! mount -o ro $rwDevice $rwDir &>/dev/null - then - Echo "Creating filesystem for RW data on $rwDevice..." - local exception_handling="false" - if ! createFilesystem $rwDevice "" "" "hybrid" $exception_handling $hybrid_fs; then - Echo "Failed to create ext3 filesystem" - return 1 - fi - checkFilesystem $rwDevice >/dev/null - fi - else - umount $rwDevice + Echo "Failed to create ${hybrid_fs} filesystem" + return 1 fi fi return 0 @@ -6519,6 +6536,7 @@ local roDevice=`echo $UNIONFS_CONFIG | cut -d , -f 2` local unionFST=`echo $UNIONFS_CONFIG | cut -d , -f 3` local prefix=/mnt + local mount_options #====================================== # load fuse module #-------------------------------------- @@ -6565,7 +6583,13 @@ if [ ! "$roDevice" = "nfs" ] && ! setupReadWrite; then return 1 fi - if ! kiwiMount "$rwDevice" "$rwDir";then + if [ "$kiwi_hybridpersistent" = "true" ];then + # When using an overlay writing to a block device safety has + # less priority over speed. If this does not match your use + # case please report an issue on the kiwi github + mount_options="-o defaults,async,relatime,nodiratime" + fi + if ! kiwiMount "$rwDevice" "$rwDir" "$mount_options";then Echo "Failed to mount read/write filesystem" return 1 fi @@ -9113,7 +9137,7 @@ local label=$(blkid $partd -s LABEL -o value) if [ "$label" = "hybrid" ];then Echo "Existing persistent hybrid partition found" - if [ $hybrid_fs = "fat" ];then + if [ "$hybrid_fs" = "fat" ] || [ "$hybrid_fs" = "exfat" ];then if ! setupHybridCowDevice;then Echo "Failed to setup hybrid cow device" Echo "Persistent writing deactivated" @@ -9165,6 +9189,7 @@ # based persistent write partitions to be created as first partition # on efi|uefi ISO hybrid images echo -e "n\np\n$pID\n\n\nw\nq" | fdisk $imageDiskDevice + blockdev --rereadpt $imageDiskDevice else createPartitionerInput \ n p:lxrw $pID . . t $pID $HYBRID_PERSISTENT_ID @@ -9189,9 +9214,12 @@ # create filesystem on write partition #-------------------------------------- local hybrid_device=$(ddn $device $pID) - local exception_handling="false" + local fs_opts + if [ "$hybrid_fs" = "ext4" ];then + fs_opts="$HYBRID_EXT4_OPTS" + fi if ! createFilesystem \ - $hybrid_device "" "" "hybrid" $exception_handling $hybrid_fs + $hybrid_device "" "" "hybrid" "false" "$hybrid_fs" "$fs_opts" then Echo "Failed to create hybrid persistent filesystem" Echo "Persistent writing deactivated" @@ -9201,7 +9229,7 @@ #====================================== # export read-write device name #-------------------------------------- - if [ $hybrid_fs = "fat" ];then + if [ "$hybrid_fs" = "fat" ] || [ "$hybrid_fs" = "exfat" ];then # The fat filesystem is not really suitable to be used as rootfs # for linux. Therefore we create a btrfs based file which we store # on the fat filesystem and loop setup it. The size of the file @@ -9228,25 +9256,41 @@ local hybrid_device=$1 mkdir -p /cow for i in 1 2 3;do - mount -L hybrid /cow && break || sleep 2 + if [ "$hybrid_fs" = "exfat" ]; then + mount $hybrid_device /cow ||\ + mount.exfat $hybrid_device /cow && break || sleep 2 + else + mount -L hybrid /cow && break || sleep 2 + fi done - if [ ! $? = 0 ];then + if ! mountpoint -q /cow; then Echo "Failed to mount hybrid persistent filesystem !" return 1 fi - if [ ! -e "/cow/cowfile" ];then - local cowsize=$(($(blockdev --getsize64 $hybrid_device) / 2)) - local exception_handling="false" - qemu-img create /cow/cowfile $cowsize + local hybrid_cow_filename="/cow/${HYBRID_PERSISTENT_FILENAME}" + if [ ! -z "$kiwi_hybridpersistent_cow_filename" ];then + hybrid_cow_filename="/cow/${kiwi_hybridpersistent_cow_filename}" + fi + if [ ! -e "$hybrid_cow_filename" ];then + # default cow filesize is half of partition's capacity + local cowsize="$(($(blockdev --getsize64 $hybrid_device) / 2))" + # but not for FAT due to the 4G file size limit + if [ "$hybrid_fs" = "fat" ] && [ "$cowsize" -gt 4294967295 ];then + cowsize=4294967295 + fi + if [ ! -z "$kiwi_hybridpersistent_filesize" ];then + cowsize=$kiwi_hybridpersistent_filesize + fi + qemu-img create "$hybrid_cow_filename" "$cowsize" if ! createFilesystem \ - "/cow/cowfile" "" "" "" $exception_handling "btrfs" + "$hybrid_cow_filename" "" "" "" "false" "ext4" "$HYBRID_EXT4_OPTS" then Echo "Failed to create hybrid persistent cow filesystem" return 1 fi fi - export HYBRID_RW=$(loop_setup /cow/cowfile) - if [ ! -e "$cowdevice" ];then + export HYBRID_RW=$(loop_setup "$hybrid_cow_filename") + if [ ! -e "$HYBRID_RW" ];then Echo "Failed to loop setup hybrid cow file !" return 1 fi @@ -9731,11 +9775,11 @@ if [ "$FSTYPE" = "reiserfs" ];then reiserfsck -y $device elif [ "$FSTYPE" = "ext2" ];then - e2fsck -p $device + e2fsck -p -f $device elif [ "$FSTYPE" = "ext3" ];then - e2fsck -p $device + e2fsck -p -f $device elif [ "$FSTYPE" = "ext4" ];then - e2fsck -p $device + e2fsck -p -f $device elif [ "$FSTYPE" = "btrfs" ];then btrfsck $device elif [ "$FSTYPE" = "xfs" ];then @@ -9910,33 +9954,39 @@ if [ ! -z "$label" ];then opts="$opts -L $label" fi + elif [ "$filesystem" = "exfat" ];then + if [ ! -z "$label" ];then + opts="$opts -n $label" + fi fi if [ "$filesystem" = "reiserfs" ];then - mkreiserfs $opts $deviceCreate $blocks 1>&2 + mkreiserfs $opts "$deviceCreate" $blocks 1>&2 elif [ "$filesystem" = "ext2" ];then - mkfs.ext2 $opts $deviceCreate $blocks 1>&2 + mkfs.ext2 $opts "$deviceCreate" $blocks 1>&2 elif [ "$filesystem" = "ext3" ];then - mkfs.ext3 $opts $deviceCreate $blocks 1>&2 + mkfs.ext3 $opts "$deviceCreate" $blocks 1>&2 elif [ "$filesystem" = "ext4" ];then - mkfs.ext4 $opts $deviceCreate $blocks 1>&2 + mkfs.ext4 $opts "$deviceCreate" $blocks 1>&2 elif [ "$filesystem" = "btrfs" ];then # delete potentially existing btrfs on the device because even # with the force option enabled mkfs.btrfs refuses to create a # filesystem if the existing metadata contains the same UUID - dd if=/dev/zero of=$deviceCreate bs=1M count=1 conv=notrunc - mkfs.btrfs $opts $deviceCreate + dd if=/dev/zero of="$deviceCreate" bs=1M count=1 conv=notrunc + mkfs.btrfs $opts "$deviceCreate" elif [ "$filesystem" = "xfs" ];then - mkfs.xfs $opts -f $deviceCreate + mkfs.xfs $opts -f "$deviceCreate" if [ ! -z "$uuid" ];then - xfs_admin -U $uuid $deviceCreate + xfs_admin -U $uuid "$deviceCreate" fi elif [ "$filesystem" = "fat" ];then - mkfs.fat $opts $deviceCreate $blocks 1>&2 + mkfs.fat $opts "$deviceCreate" $blocks 1>&2 elif [ "$filesystem" = "ntfs" ];then - mkfs.ntfs $opts $deviceCreate $blocks 1>&2 + mkfs.ntfs $opts "$deviceCreate" $blocks 1>&2 + elif [ "$filesystem" = "exfat" ];then + mkfs.exfat $opts "$deviceCreate" 1>&2 else # use ext3 by default - mkfs.ext3 $opts $deviceCreate $blocks 1>&2 + mkfs.ext3 $opts "$deviceCreate" $blocks 1>&2 fi if [ $exception_handling = "false" ];then return $? @@ -11384,12 +11434,12 @@ #-------------------------------------- function loop_setup { local IFS=$IFS_ORIG - local target=$1 + local target="$@" local logical_sector_size if [ ! -z "$kiwi_target_blocksize" ];then logical_sector_size="-L $kiwi_target_blocksize" fi - local loop=$(losetup $logical_sector_size -f --show $target) + local loop=$(losetup $logical_sector_size -f --show "$target") if [ ! -e "$loop" ];then return 1 fi @@ -11400,8 +11450,8 @@ #-------------------------------------- function loop_delete { local IFS=$IFS_ORIG - local target=$1 - losetup -d $target + local target="$@" + losetup -d "$target" } #====================================== # startMultipathd diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/kiwi/modules/KIWISchema.rnc new/kiwi/modules/KIWISchema.rnc --- old/kiwi/modules/KIWISchema.rnc 2015-12-23 09:12:14.000000000 +0100 +++ new/kiwi/modules/KIWISchema.rnc 2016-01-14 12:50:28.000000000 +0100 @@ -1783,7 +1783,7 @@ ## hybrid ISO is used as disk on e.g a USB Stick. By default ## the btrfs filesystem is used attribute hybridpersistent_filesystem { - "btrfs" | "fat" | "ext4" | "xfs" + "btrfs" | "fat" | "exfat" | "ext4" | "xfs" } k.type.image.attribute = ## Specifies the image type diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/kiwi/modules/KIWISchema.rng new/kiwi/modules/KIWISchema.rng --- old/kiwi/modules/KIWISchema.rng 2015-12-23 09:12:14.000000000 +0100 +++ new/kiwi/modules/KIWISchema.rng 2016-01-14 12:50:28.000000000 +0100 @@ -2403,6 +2403,7 @@ <choice> <value>btrfs</value> <value>fat</value> + <value>exfat</value> <value>ext4</value> <value>xfs</value> </choice>
