Bug#486926: Installed system does not have correct permissions on directories
Joey Hess wrote: As a bonus, before reading your reply I had re-implemented this functionality in C which sped it up by a factor of 4 - using tar seems to have the same effect. Using C is an interesting idea.. yes. using tar from the live-system would be unfortunate, as it highers the requirements on the live-system (yes, I'm aware that tar is of priority required). currently, live-installer just 'dumps' any live-system to target, which makes it universally usable, not just for pure debian. OTOH having additional C code isn't much of a problem, since the udeb is already arch dependent anyway. -- Address:Daniel Baumann, Burgunderstrasse 3, CH-4562 Biberist Email: [EMAIL PROTECTED] Internet: http://people.panthera-systems.net/~daniel-baumann/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#486926: Installed system does not have correct permissions on directories
Joey Hess wrote: something seems to block the filename output from tar, causing the progress bar to sit at 2% until very near the end - it then shoots up quite quickly. Right, I guess I'd need to use a read loop to get the lines one at a time rather than having it buffer. Updated patch attached. That fixes that problem, but now the read command started by db_progress STEP 1 is reading a filename instead of the the STEP value, resulting in Invalid value: ./bin/, and the installation failing. I tried a few fd/fifo redirection tricks, but couldn't get the right combination..any ideas? I also benchmarked this tar method and it seems to be about three or four times faster than my C version, putting it approximately 10 (!) times faster than the current shell version. (I agree with Daniel's remark regarding not depending on 'tar' though -- perhaps live-installer itself could embed or depend on a functioning tar binary?) Regards, -- Chris Lamb, UK [EMAIL PROTECTED] GPG: 0x634F9A20 signature.asc Description: PGP signature
Bug#486926: Installed system does not have correct permissions on directories
tags 486926 + patch
thanks
Chris Lamb wrote:
I tried a few fd/fifo redirection tricks, but couldn't get the right
combination..any ideas?
Got it, hurrah. Updated and tested patch attached.
Regards,
--
Chris Lamb, UK [EMAIL PROTECTED]
GPG: 0x634F9A20
From 8d1275ccb68117d1f288adf4691f255b09bbe71a Mon Sep 17 00:00:00 2001
From: Chris Lamb [EMAIL PROTECTED]
Date: Thu, 19 Jun 2008 16:10:42 +0100
Subject: [PATCH] I suggest getting rid of this mkdir/rm/cp nonsense and just
pipe tar to
tar. d-i busybox tar cannot create tar archives, but that's ok, we have
a live filesystem with a fullfledged tar on it available to use.
---
packages/live-installer/debian/changelog |2 +
packages/live-installer/debian/postinst | 45 +
2 files changed, 28 insertions(+), 19 deletions(-)
diff --git a/packages/live-installer/debian/changelog
b/packages/live-installer/debian/changelog
index 6b6de43..4cfaac7 100644
--- a/packages/live-installer/debian/changelog
+++ b/packages/live-installer/debian/changelog
@@ -2,6 +2,8 @@ live-installer (6) UNRELEASED; urgency=low
* Bail-out much earlier if a filesystem image cannot be found (Chris Lamb)
Closes: #486184
+ * Copy files using tar, avoids permissions problems and other nonsense.
+Closes: #486926
-- Joey Hess [EMAIL PROTECTED] Wed, 18 Jun 2008 21:45:14 -0400
diff --git a/packages/live-installer/debian/postinst
b/packages/live-installer/debian/postinst
index c9817f9..bed9f09 100755
--- a/packages/live-installer/debian/postinst
+++ b/packages/live-installer/debian/postinst
@@ -40,25 +40,32 @@ install_live_system () {
COUNT=0
OLD_IFS=$IFS
IFS=$NEWLINE
- for item in `find .`; do
- # We need to be ensure it's not a symbolic link
otherwise
- # it breaks links for directories.
- if [ -d $item ] [ ! -h $item ]; then
- mkdir -p /target/$item
- else
- mkdir -p /target/$(dirname $item)
- rm -f /target/$item
- cp -a $item /target/$item
- fi
-
- COUNT=$(($COUNT + 1))
- CURRENT=$(($COUNT * 100 / $STEPS))
-
- [ x$CURRENT = x$LAST_UPDATE ] continue
-
- LAST_UPDATE=$CURRENT
- db_progress STEP 1
- done
+ mkdir -p /target
+ # use tar from inside the live filesystem to create
+ # the tarball, because busybox tar in d-i does not
+ # support creating tarballs.
+ #
+ # The --exclude is a paranoia measure, in case this program
+ # is running from the toplevel of a live filesystem,
+ # which is not normally the case.
+
+ exec 40
+ chroot . tar c . --exclude=target | \
+ (chdir /target tar xv) | \
+ (
+ while read line exec 50; do
+ COUNT=$(($COUNT + 1))
+ CURRENT=$(($COUNT * 100 / $STEPS))
+
+ [ x$CURRENT = x$LAST_UPDATE ] continue
+
+ LAST_UPDATE=$CURRENT
+ exec 4
+ db_progress STEP 1
+ exec 5
+ done
+ )
+ exec 4- 5-
IFS=$OLD_IFS
done
--
1.5.5.4
signature.asc
Description: PGP signature
Bug#486926: Installed system does not have correct permissions on directories
Chris Lamb wrote:
That fixes that problem, but now the read command started by db_progress
STEP 1 is reading a filename instead of the the STEP value, resulting in
Invalid value: ./bin/, and the installation failing.
I tried a few fd/fifo redirection tricks, but couldn't get the right
combination..any ideas?
I still haven't been able to fully test this, sorry, but I think this
should work. stdin (debconf) is redirected to fd 4 and sent into the
dh_progress command, then restored at the end.
I also benchmarked this tar method and it seems to be about three or four
times faster than my C version, putting it approximately 10 (!) times faster
than the current shell version. (I agree with Daniel's remark regarding not
depending on 'tar' though -- perhaps live-installer itself could embed or
depend on a functioning tar binary?)
Or we could take the small size hit and turn on tar creation in busybox
tar.
However, if this patch fixes the immediate problem, I'd be inclined to
look at supporting target systems that don't have a working tar as a
wishlist feature that could be implemented later. d-i requires a great
deal from the target system after it's unpacked, including a fully working
apt, which implies a working dpkg, which implies a working tar..
--
see shy jo
From bf5104387f5a22c3119bd9d9dcc0d1adf0762f03 Mon Sep 17 00:00:00 2001
From: Joey Hess [EMAIL PROTECTED]
Date: Wed, 18 Jun 2008 22:16:50 -0400
Subject: [PATCH] Copy files using tar, avoids permissions problems and other nonsense
I suggest getting rid of this mkdir/rm/cp nonsense and just pipe tar to
tar. d-i busybox tar cannot create tar archives, but that's ok, we have
a live filesystem with a fullfledged tar on it available to use.
There is some FD trickery needed by the progress update loop since it is
piped filenames on stdin. Real stdin (debconf) is redirected to fd 4
and sent into the dh_progress command, then restored at the end. fd 4 should
be safe with both cdebconf and debconf as they use 3 and 5, respectively.
This patch has not yet been tested. (take 3)
---
packages/live-installer/debian/changelog |7 +
packages/live-installer/debian/postinst | 43 --
2 files changed, 30 insertions(+), 20 deletions(-)
diff --git a/packages/live-installer/debian/changelog b/packages/live-installer/debian/changelog
index 2ad2913..9b03bc0 100644
--- a/packages/live-installer/debian/changelog
+++ b/packages/live-installer/debian/changelog
@@ -1,3 +1,10 @@
+live-installer (6) UNRELEASED; urgency=low
+
+ * Copy files using tar, avoids permissions problems and other nonsense.
+Closes: #486926
+
+ -- Joey Hess [EMAIL PROTECTED] Wed, 18 Jun 2008 21:23:06 -0400
+
live-installer (5) unstable; urgency=low
[ Updated translations ]
diff --git a/packages/live-installer/debian/postinst b/packages/live-installer/debian/postinst
index cd3313d..5bd585e 100755
--- a/packages/live-installer/debian/postinst
+++ b/packages/live-installer/debian/postinst
@@ -36,26 +36,29 @@ install_live_system () {
COUNT=0
OLD_IFS=$IFS
- IFS=$NEWLINE
- for item in `find .`; do
- # We need to be ensure it's not a symbolic link otherwise
- # it breaks links for directories.
- if [ -d $item ] [ ! -h $item ]; then
-mkdir -p /target/$item
- else
-mkdir -p /target/$(dirname $item)
-rm -f /target/$item
-cp -a $item /target/$item
- fi
-
- COUNT=$(($COUNT + 1))
- CURRENT=$(($COUNT * 100 / $STEPS))
-
- [ x$CURRENT = x$LAST_UPDATE ] continue
-
- LAST_UPDATE=$CURRENT
- db_progress STEP 1
- done
+ mkdir -p /target
+ # use tar from inside the live filesystem to create
+ # the tarball, because busybox tar in d-i does not
+ # support creating tarballs.
+ #
+ # The --exclude is a paranoia measure, in case this program
+ # is running from the toplevel of a live filesystem,
+ # which is not normally the case.
+ exec 40
+ chroot . tar c . --exclude=target | \
+ (chdir /target tar xv) | \
+ (
+ while read line; do
+COUNT=$(($COUNT + 1))
+CURRENT=$(($COUNT * 100 / $STEPS))
+
+[ x$CURRENT = x$LAST_UPDATE ] continue
+
+LAST_UPDATE=$CURRENT
+db_progress STEP 1 4
+ done
+ )
+ exec 04
IFS=$OLD_IFS
done
--
1.5.5.4
signature.asc
Description: Digital signature
Bug#486926: Installed system does not have correct permissions on directories
Chris Lamb wrote: tags 486926 + patch thanks Chris Lamb wrote: I tried a few fd/fifo redirection tricks, but couldn't get the right combination..any ideas? Got it, hurrah. Updated and tested patch attached. Huh, it looks like I was at least on the right track with my patch.. + exec 40 + while read line exec 50; do + exec 4 + db_progress STEP 1 + exec 5 + exec 4- 5- If I'm managing to read this right, you use fd 5 only to store fd 0 temporarily. If so, jusr db_progress STEP 1 4 should work just as well and less messily, like in my patch. -- see shy jo signature.asc Description: Digital signature
Bug#486926: Installed system does not have correct permissions on directories
Joey Hess wrote: If so, jusr db_progress STEP 1 4 should work just as well and less messily, like in my patch. Your patch works. Please disregard mine.. many thanks. Regards, -- Chris Lamb, UK [EMAIL PROTECTED] GPG: 0x634F9A20 signature.asc Description: PGP signature
Bug#486926: Installed system does not have correct permissions on directories
Chris Lamb [EMAIL PROTECTED] writes: Joey Hess wrote: If so, jusr db_progress STEP 1 4 should work just as well and less messily, like in my patch. Your patch works. Please disregard mine.. many thanks. Joey, could you please commit it? -- O T A V I OS A L V A D O R - E-mail: [EMAIL PROTECTED] UIN: 5906116 GNU/Linux User: 239058 GPG ID: 49A5F855 Home Page: http://otavio.ossystems.com.br - Microsoft sells you Windows ... Linux gives you the whole house. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#486926: Installed system does not have correct permissions on directories
Package: live-installer Severity: normal Version: 5 After installation via live-installer, permisions on directories are all 0755. This is especially noticable on /tmp, /var/mail, /etc/ssl/private, etc. Regards, -- Chris Lamb, UK [EMAIL PROTECTED] GPG: 0x634F9A20 From 1ab50c8fc84f05c6b12ee6c8ba470cb25d682025 Mon Sep 17 00:00:00 2001 From: Chris Lamb [EMAIL PROTECTED] Date: Sat, 14 Jun 2008 06:40:13 +0100 Subject: [PATCH] Override live-image network data info from d-i instead of removing it After installation via live-installer, one is left with a system without networking dispite netcfg configuring it correctly. This patch copies d-i networking settings (if they exist) to the target. This operation is already performed a base-installer.d pre-installation hook, but the changes are overridden during the extraction of the live filesystem image. An alternative to this patch would be to run the pre- installation hooks post-extraction, but that seems to be asking for trouble. Signed-off-by: Chris Lamb [EMAIL PROTECTED] --- packages/live-installer/live-installer.d/network | 10 -- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/packages/live-installer/live-installer.d/network b/packages/live-installer/live-installer.d/network index 27863b6..93e461a 100755 --- a/packages/live-installer/live-installer.d/network +++ b/packages/live-installer/live-installer.d/network @@ -1,4 +1,10 @@ #!/bin/sh -e -# Remove live-cd network data -rm -f /target/etc/hostname /target/etc/hosts /target/etc/network/interfaces +# Override live-cd network data info from d-i +for file in /etc/network/interfaces /etc/networks /etc/hostname /etc/resolv.conf /etc/hosts; do + rm -f /target/$file + if [ -f $file ]; then + mkdir /target/$(dirname $file) -p + cp $file /target/$file + fi +done -- 1.5.5.4 signature.asc Description: PGP signature
Bug#486926: Installed system does not have correct permissions on directories
Chris Lamb wrote:
Package: live-installer
Severity: normal
Version: 5
After installation via live-installer, permisions on directories are all
0755. This is especially noticable on /tmp, /var/mail, /etc/ssl/private, etc.
If that's the case, the bug should probably be RC; a system with such
busted permissions is generally broken.
And it looks like the bug is that it simply runs mkdir -p before copying
w/o trying to preserve dir perms at all:
if [ -d $item ] [ ! -h $item ]; then
mkdir -p /target/$item
else
mkdir -p /target/$(dirname $item)
rm -f /target/$item
cp -a $item /target/$item
fi
The thing to do seems to be to use tar instead.
I cannot test the attached patch easily, since I have no live CDs to use
it with. Can someone please give it a try?
--
see shy jo
From 8b0ed77c1eeb00076ce97e553d1ed21a9ad39ba8 Mon Sep 17 00:00:00 2001
From: Joey Hess [EMAIL PROTECTED]
Date: Wed, 18 Jun 2008 22:16:50 -0400
Subject: [PATCH] Copy files using tar, avoids permissions problems and other nonsense
I suggest getting rid of this mkdir/rm/cp nonsense and just pipe tar to
tar. d-i busybox tar cannot create tar archives, but that's ok, we have
a live filesystem with a fullfledged tar on it available to use.
This patch has not yet been tested.
---
packages/live-installer/debian/changelog |7 +++
packages/live-installer/debian/postinst | 20 +---
2 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/packages/live-installer/debian/changelog b/packages/live-installer/debian/changelog
index 2ad2913..9b03bc0 100644
--- a/packages/live-installer/debian/changelog
+++ b/packages/live-installer/debian/changelog
@@ -1,3 +1,10 @@
+live-installer (6) UNRELEASED; urgency=low
+
+ * Copy files using tar, avoids permissions problems and other nonsense.
+Closes: #486926
+
+ -- Joey Hess [EMAIL PROTECTED] Wed, 18 Jun 2008 21:23:06 -0400
+
live-installer (5) unstable; urgency=low
[ Updated translations ]
diff --git a/packages/live-installer/debian/postinst b/packages/live-installer/debian/postinst
index cd3313d..2608e00 100755
--- a/packages/live-installer/debian/postinst
+++ b/packages/live-installer/debian/postinst
@@ -37,17 +37,15 @@ install_live_system () {
COUNT=0
OLD_IFS=$IFS
IFS=$NEWLINE
- for item in `find .`; do
- # We need to be ensure it's not a symbolic link otherwise
- # it breaks links for directories.
- if [ -d $item ] [ ! -h $item ]; then
-mkdir -p /target/$item
- else
-mkdir -p /target/$(dirname $item)
-rm -f /target/$item
-cp -a $item /target/$item
- fi
-
+ mkdir -p /target
+ # use tar from inside the live filesystem to create
+ # the tarball, because busybox tar in d-i does not
+ # support creating tarballs.
+ #
+ # The --exclude is a paranoia measure, in case this program
+ # is running from the toplevel of a live filesystem,
+ # which is not normally the case.
+ for line in $(chroot . tar c . --exclude=target | (chdir /target tar xv)); do
COUNT=$(($COUNT + 1))
CURRENT=$(($COUNT * 100 / $STEPS))
--
1.5.5.4
signature.asc
Description: Digital signature
Bug#486926: Installed system does not have correct permissions on directories
Joey Hess wrote: The thing to do seems to be to use tar instead. I cannot test the attached patch easily, since I have no live CDs to use it with. Just tried it here. Whilst the files are copied with the correct permissions (yay), something seems to block the filename output from tar, causing the progress bar to sit at 2% until very near the end - it then shoots up quite quickly. As a bonus, before reading your reply I had re-implemented this functionality in C which sped it up by a factor of 4 - using tar seems to have the same effect. Regards, -- Chris Lamb, UK [EMAIL PROTECTED] GPG: 0x634F9A20 signature.asc Description: PGP signature
Bug#486926: Installed system does not have correct permissions on directories
Chris Lamb wrote:
Just tried it here. Whilst the files are copied with the correct permissions
(yay), something seems to block the filename output from tar, causing the
progress bar to sit at 2% until very near the end - it then shoots up quite
quickly.
Right, I guess I'd need to use a read loop to get the lines one at a
time rather than having it buffer. Updated patch attached.
As a bonus, before reading your reply I had re-implemented this
functionality in C which sped it up by a factor of 4 - using tar seems to
have the same effect.
Using C is an interesting idea..
--
see shy jo
From 0b30e08a28fb8bceed82bf01c8c2f18eedf30687 Mon Sep 17 00:00:00 2001
From: Joey Hess [EMAIL PROTECTED]
Date: Wed, 18 Jun 2008 22:16:50 -0400
Subject: [PATCH] Copy files using tar, avoids permissions problems and other nonsense
I suggest getting rid of this mkdir/rm/cp nonsense and just pipe tar to
tar. d-i busybox tar cannot create tar archives, but that's ok, we have
a live filesystem with a fullfledged tar on it available to use.
This patch has not yet been tested. (take 2)
---
packages/live-installer/debian/changelog |7 ++
packages/live-installer/debian/postinst | 43 +-
packages/live-installer/debian/postinst.orig | 123 ++
3 files changed, 152 insertions(+), 21 deletions(-)
create mode 100755 packages/live-installer/debian/postinst.orig
diff --git a/packages/live-installer/debian/changelog b/packages/live-installer/debian/changelog
index 2ad2913..9b03bc0 100644
--- a/packages/live-installer/debian/changelog
+++ b/packages/live-installer/debian/changelog
@@ -1,3 +1,10 @@
+live-installer (6) UNRELEASED; urgency=low
+
+ * Copy files using tar, avoids permissions problems and other nonsense.
+Closes: #486926
+
+ -- Joey Hess [EMAIL PROTECTED] Wed, 18 Jun 2008 21:23:06 -0400
+
live-installer (5) unstable; urgency=low
[ Updated translations ]
diff --git a/packages/live-installer/debian/postinst b/packages/live-installer/debian/postinst
index cd3313d..5dc222b 100755
--- a/packages/live-installer/debian/postinst
+++ b/packages/live-installer/debian/postinst
@@ -36,27 +36,28 @@ install_live_system () {
COUNT=0
OLD_IFS=$IFS
- IFS=$NEWLINE
- for item in `find .`; do
- # We need to be ensure it's not a symbolic link otherwise
- # it breaks links for directories.
- if [ -d $item ] [ ! -h $item ]; then
-mkdir -p /target/$item
- else
-mkdir -p /target/$(dirname $item)
-rm -f /target/$item
-cp -a $item /target/$item
- fi
-
- COUNT=$(($COUNT + 1))
- CURRENT=$(($COUNT * 100 / $STEPS))
-
- [ x$CURRENT = x$LAST_UPDATE ] continue
-
- LAST_UPDATE=$CURRENT
- db_progress STEP 1
- done
- IFS=$OLD_IFS
+ mkdir -p /target
+ # use tar from inside the live filesystem to create
+ # the tarball, because busybox tar in d-i does not
+ # support creating tarballs.
+ #
+ # The --exclude is a paranoia measure, in case this program
+ # is running from the toplevel of a live filesystem,
+ # which is not normally the case.
+ chroot . tar c . --exclude=target | \
+ (chdir /target tar xv) | \
+ (
+ while read line; do
+COUNT=$(($COUNT + 1))
+CURRENT=$(($COUNT * 100 / $STEPS))
+
+[ x$CURRENT = x$LAST_UPDATE ] continue
+
+LAST_UPDATE=$CURRENT
+db_progress STEP 1
+ done
+ IFS=$OLD_IFS
+ )
done
# if we're dumping it, we need to set boot=live
diff --git a/packages/live-installer/debian/postinst.orig b/packages/live-installer/debian/postinst.orig
new file mode 100755
index 000..e1e331b
--- /dev/null
+++ b/packages/live-installer/debian/postinst.orig
@@ -0,0 +1,123 @@
+#! /bin/sh
+set -e
+
+. /usr/share/debconf/confmodule
+db_capb backup
+
+. /usr/lib/base-installer/library.sh
+
+NEWLINE=
+
+
+db_input low live-installer/mode || true
+db_go || exit 10 # back to menu
+db_get live-installer/mode
+mode=$RET
+
+install_live_system () {
+ # Look at
+ PLACES=
+
+ # Load filesystem support
+ for script in $(ls /lib/live-installer/*); do
+ . $script
+ done
+
+ for place in $PLACES; do
+ [ ! -e $place ] continue
+
+ SUPPORT=$(echo $place | sed 's,.*\.\(.*\)$,\1,g')
+ info Using $SUPPORT support for $place
+
+ eval ${SUPPORT}_prepare
+ STEPS=$(eval ${SUPPORT}_count)
+
+ db_progress INFO live-installer/progress/copying
+
+ COUNT=0
+ OLD_IFS=$IFS
+ IFS=$NEWLINE
+ for item in `find .`; do
+ # We need to be ensure it's not a symbolic link otherwise
+ # it breaks links for directories.
+ if [ -d $item ] [ ! -h $item ]; then
+mkdir -p /target/$item
+ else
+mkdir -p /target/$(dirname $item)
+rm -f /target/$item
+cp -a $item /target/$item
+ fi
+
+ COUNT=$(($COUNT + 1))
+ CURRENT=$(($COUNT * 100 / $STEPS))
+
+ [ x$CURRENT = x$LAST_UPDATE ] continue
+
+ LAST_UPDATE=$CURRENT
+ db_progress STEP 1
+ done
+ IFS=$OLD_IFS
+ done
+
+ # if we're dumping it, we need to set boot=live
+ if [ $mode = live ]; then
+ # set the init script to use
+ if [ -d /cdrom/casper
