Hi,
I'll try to describe my efforts to integrate http filesystem into casper.
There is more work to be done and help is welcome.
First, bear in mind the following when dealing with http filesystems:
1. AFAIK there are two implementations for http filesystems:
httpfs<http://httpfs.sourceforge.net/>and
http-fuse-cloop<http://unit.aist.go.jp/itri/knoppix/http-fuse/index-en.html>.
Both projects are not part of Debian and IMHO are still immature and not
ready for production use. There are several filesystems out there that can
be used as well, such as davfs and curlftpfs.
2. DHCP is a must, similar to nfsroot and cifsroot.
3. As you can imagine, the performance can be really bad when the network
connection is not fast. There are few ways to optimize the performance such
as local disk caching, preloading of blocks/files and compression. httpfs
lacks the disk caching and preloading.
4. These filesystems are bloat because of their dependencies, namely glibc
and fuse. Inserting them into the initramfs, makes the cpio image bigger
(important in some scenarios).
5. Both filesystem does not support plain directories (this is another
feature I would like to see sometime). httpfs hooks to a single image file
such as iso or squashfs, while http-fuse-cloop uses an index file and a set
of blocks files.
6. This feature does not replace the boot itself. The system must be booted
from a local device or a PXE server! I don't know any BIOS that fetches boot
images over http. Maybe linuxbios? :)
We (neocleus.com) decided to use http fuse cloop (fs_wrapper) for the first
version of the product, so we can have disk caching and preloading. In the
future we hope to use a better httpfs + squashfs. Here's the code we've used
in casper:
* Added this function:
do_httpmount() {
rc=1
if [ -x "/sbin/fs_wrapper" ]; then
[ "$quiet" != "y" ] && log_begin_msg "Trying fs_wrapper -f
${NFSROOT} ${mountpoint}"
modprobe "${MP_QUIET}" fuse
mkdir -p /cow/blocks
fs_wrapper -f "${mountpoint}" "${NFSROOT}" &
for i in 0 1 2 3 4 5 6 7 8 9 a b c d e f 10 11 12 13; do
if [ -f "${mountpoint}"/* ]; then
rc=0
break
else
log_begin_msg "waiting for fs_wrapper:"$i
sleep 1
fi
done
fi
return ${rc}
}
* Changed netboot function to call do_httpmount instead of cifs - an ugly
hack ;)
* Added support for cloop which is necessary when using http-fuse-cloop, see
the attached patch.
* Added this code to the end of casper:
+++ #ugly hack to keep fs_wrapper running...
+++ sed -i 's/killall5/ps axwww \| grep -v fs_wrapper \| grep -v PID \|
grep -v rc \| grep -v init \| awk "\{print \\\$1\}" \| xargs +++ -n1
kill/g' /root/etc/init.d/sendsigs
+++ sed -i 's/ES\=\$?/ES\=\$?;\/usr\/bin\/killall -15 fs_wrapper;
\/bin\/sleep 3/' /root/etc/init.d/umountroot
exec 1>&6 6>&-
exec 2>&7 7>&-
cp casper.log "${rootmnt}/var/log/"
}
* Switched losetup! The current casper uses "losetup" which is the one
supplied from busybox. This losetup didn't work well for me. Therefore, I
switch to /sbin/losetup which is included in the first place. Attached a
patch for that change. Again this patch is required only for http-fuse
cloop.
* Last change was to add cloop, fuse and fs_wrapper modules to the
hooks/casper file. Make sure to have those modules in the live image.
As I wrote earlier, when the kernel switches from initramfs to the real
filesystem, run-init kills all the processes.
This hacky sed suppose to prevent the kill of the http filesystem.
From my experiments, at the initramfs phase, the filesystem works. When
run-init starts, the system just hangs. Maybe the sed script is not correct
or not enough. Any suggestions/comments/questions are welcome.
TODO list:
1. Make it work :) seriously - just solve the existing problems.
2. Improve live-initramfs handling/parsing. What you call http:// httpfs://
cifs:// nfs:// support...
3. Enhance httpfs:
3.1. Disk caching
3.2 Preloading (boot profile, like accelerated knoppix)
3.3 Load balancing - allow simultaneous connections to multiple mirrors
3.4 Network recovery - do not hang/panic when there's no network connection
and be able to continue working when the connection is back again.
3.5 Sparse image (squashfs feature) and plain directories
And there's probably more...
Cheers,
Hadar
On 5/24/07, RJ Ent. <[EMAIL PROTECTED]> wrote:
Greetings,
I would like to request a feature for netbooting, namely to have the
ability be able to get the netboot folder via http or even https.
Also then to consider the netboot param to act as examples below:
netboot=nfs://xxx.xxx.xxx.xxx/path
netboot=cifs://xxx.xxx.xxx.xxx/path
netboot=http://xxx.xxx.xxx.xxx/path
netboot=https://xxx.xxx.xxx.xxx/path
I am unsure how much work this would take but I think having the ability
to make a boot cd (or pxe setup or whatever) and just type in the web
address that you wanted it to continue booting from. Then at the server
you could just tune the netboot folders shared out via http. Not to
mention a secure session via https.
Ok ideas and petition for the feature welcome.
Thx.
----------
RJEnt. - Linux Solutions & Consulting
_______________________________________________
Debian-live-devel mailing list
[email protected]
http://lists.alioth.debian.org/mailman/listinfo/debian-live-devel
--- /home/hadar/dev/casper/casper-1.81+debian/scripts/casper
+++ /home/hadar/dev/neolive/casper/casper-1.81+debian/scripts/casper
@@ -123,6 +123,7 @@
path=$1
if [ -d "$path/casper" ]; then
if [ "$(echo $path/casper/*.squashfs)" != "$path/casper/*.squashfs" ] ||
+ [ "$(echo $path/casper/*.cloop)" != "$path/casper/*.cloop" ] ||
[ "$(echo $path/casper/*.ext2)" != "$path/casper/*.ext2" ] ||
[ "$(echo $path/casper/*.dir)" != "$path/casper/*.dir" ]; then
return 0
@@ -133,6 +134,9 @@
get_backing_device() {
case "$1" in
+ *.cloop)
+ echo $(setup_loop "$1" "cloop" "/sys/block/cloop*" '0' "${LIVEMEDIA_ENCRYPTION}")
+ ;;
*.squashfs|*.ext2)
echo $(setup_loop "$1" "loop" "/sys/block/loop*" '0' "${LIVEMEDIA_ENCRYPTION}")
;;
@@ -160,6 +164,7 @@
rootmnt="$2"
if match_files_in_dir "$directory/casper/*.squashfs" ||
match_files_in_dir "$directory/casper/*.ext2" ||
+ match_files_in_dir "$directory/casper/*.cloop" ||
match_files_in_dir "$directory/casper/*.dir"; then
setup_unionfs "$directory/casper" "$rootmnt"
else
@@ -418,7 +423,7 @@
done
else
# If ${MODULE}.lst does not exist, create a list of images
- for image_type in "ext2" "squashfs" "dir"; do
+ for image_type in "ext2" "squashfs" "cloop" "dir"; do
for image in "${image_directory}"/*."${image_type}"; do
if [ -e "${image}" ]; then
image_string="${image_string} ${image}";
--- /home/hadar/dev/casper/casper-1.81+debian/scripts/casper
+++ /home/hadar/dev/neolive/casper/casper-1.81+debian/scripts/casper
@@ -15,6 +15,8 @@
USERFULLNAME="Live session user"
HOSTNAME="live"
BUILD_SYSTEM="Custom"
+
+losetup="/sbin/losetup"
mkdir -p $mountpoint
@@ -322,7 +324,7 @@
rmdir "${tomount}"
if echo ${fromdev} | grep -qs loop; then
- losetup -d "${fromdev}"
+ ${losetup} -d "${fromdev}"
fi
return 0
else
@@ -365,7 +367,7 @@
if [ -b ${dev} ]; then
if echo "${dev}" | grep -qs loop; then
# strange things happens, user confused?
- snaploop=$( losetup ${dev} | awk '{print $3}' | tr -d '()' )
+ snaploop=$( ${losetup} ${dev} | awk '{print $3}' | tr -d '()' )
snapfile=$(basename ${snaploop})
snapdev=$(cat /proc/mounts | awk '{print $2,$1}' | grep -es "^$( dirname ${snaploop} )" | cut -f2 -d ' ')
else
@@ -527,7 +529,7 @@
fi
if [ -n "${LIVEMEDIA_OFFSET}" ]; then
- losetup -d "${loopdevname}"
+ ${losetup} -d "${loopdevname}"
fi
return 1
}
_______________________________________________
Debian-live-devel mailing list
[email protected]
http://lists.alioth.debian.org/mailman/listinfo/debian-live-devel
