Re: Booting Debian-live-501 over HTTP
2009/9/26 Pravin shindepra...@gmail.com: Hi, 2009/9/26 Michal Suchanek hramr...@centrum.cz: What libraries did you copy? Here are the libraries copied. http://boot.kernel.org/bko/live/debian/initmod/libs/ There might be few more libraries copied than needed. I guess you need some of the libnss stuff. Probably libnss_dns and possibly libnss_compat. Try adding them to the live hook tokgether with the iscsiadm binary. The rest should be copied automatically as required by the binary copy function. You would have to relink every binary copied to initramfs with ulibc. This is not feasible because you would have to download the sources of appropriate packages and rebuild them in a non-standard environment in which many packages may fail to build simply because ulibc does not provide all features they require. Sorry, but I am not sure if I get your point. If I statically link one program with ulibc, why do I need to worry about other binaries? If you just compile a resolver and run anything that does not look like an IP through it that would also work. However, adding the glibc bit for resolving through DNS should work too, and without any additional scripting. If you are using debial-live to build a netbootable Ubuntu then this is probably the right place. I recall seeing some code for Ubuntu in live-initramfs. networkManager and networking scripts from Ubuntu are proving problems for netbooting. Netbooting works fine when I delete/disable these scripts. but when these scripts are enabled, netbooting never succeeds. I was wondering, where can I get help on these scripts. Probably at Ubuntu side because in Debian network manager is not required part of the system (and there should be a package called something like administrator-sanity which conflicts with network manager and a few others). The networking scripts in Debian do fail sometimes, try removing all the dhcp pre-up (or something like that) scripts which are called with the network down and may cause a short lockup or complete boot failure. If you do not know which ones just remove all of them to be sure, they probably don't do anything vital for a live system. They are useful for adapting services to changes in network configurations such as when connecting a notebook to different networks. HTH Michal -- To UNSUBSCRIBE, email to debian-live-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Booting Debian-live-501 over HTTP
Hi, 2009/9/26 Michal Suchanek hramr...@centrum.cz: debian-live should use the address specified with ip= and try dhcp otherwise. There are two reasons for this. Trying dhcp on a network that does not have a dhcp server takes a long time because the dhcp client would go through several increasing timeouts before giving up. The ip= parameter must be set in a specific way for each statically configures network, there is no default that can be used in case dhcp fails. Thanks for clarification. The reason I was thinking about that approach becase, the ip= parameter does not include dns entry, that needs to be passed separately, by using something like dns=. This approach is fine when someone who knows network setup is setting up boot parameter manually. But it does not scale well for automated configuration. I use pxelinux to set ip related parameters based on the network configuration discovered by gpxe. This is done by using pxelinux option IPAPPEND http://syslinux.zytor.com/wiki/index.php/SYSLINUX#IPAPPEND_flag_val_.5BPXELINUX_only.5D This mechanism does not append dns information to boot parameters. Simplest way out from here will be to modify pxelinux to include dns information. Trying to get that approved at syslinux. What libraries did you copy? Here are the libraries copied. http://boot.kernel.org/bko/live/debian/initmod/libs/ There might be few more libraries copied than needed. You would have to relink every binary copied to initramfs with ulibc. This is not feasible because you would have to download the sources of appropriate packages and rebuild them in a non-standard environment in which many packages may fail to build simply because ulibc does not provide all features they require. Sorry, but I am not sure if I get your point. If I statically link one program with ulibc, why do I need to worry about other binaries? I do not plan to replace all libraries of initramfs with ulibc. Only the statically linked dns-lookup program will not use them. As per my understanding, the httpfs binary that I am using is doing the same. It is the only binary in initramfs which can do dnslookup properly. I need similar binary, which will do similar dnslookup and output the IP. I guess copying the required library from the chroot is the right way. This can be easily repeated for every debian-live build. If you are using debial-live to build a netbootable Ubuntu then this is probably the right place. I recall seeing some code for Ubuntu in live-initramfs. networkManager and networking scripts from Ubuntu are proving problems for netbooting. Netbooting works fine when I delete/disable these scripts. but when these scripts are enabled, netbooting never succeeds. I was wondering, where can I get help on these scripts. Thanks Michal Regards -- Pravin -- To UNSUBSCRIBE, email to debian-live-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Booting Debian-live-501 over HTTP
2009/9/9 Pravin shindepra...@gmail.com:
Hi,
2009/9/7 Michal Suchanek hramr...@centrum.cz:
All moinpoints have to be moved under /root for run-init to work so I
added one more move since unpacking an iso requires an additional
mountpoint.
I tested your patch.
It works :-)
There was only small timing issue as after doing modprobe fuse
the /dev/fuse was not appearing immediately.
So, I have added small code which will wait till /dev/fuse actually appears.
I have never seen this issue on various systems I am booting for hdd
testing/cloning.
What system did you test on?
Other than that, I have cleaned up the iscsi related patch.
Removed all the comments from iscsi configuration files.
The iscsi initiator name can be passed with boot parameter iname.
In case this parameter is not present, then defualt name
preconfigured in initramfs is used.
You still hardcode the volume name which is more of a problem than the
initiator name which is irrelevant.
I create an initiator name from an arbitrary string and mac address
but it seems live-initramfs has md5sum so it should be possible to
generate hashes of arbitrary data to stick at the end of the template.
in addition to DHCP networking, current patch contains support for
static IP booting.
User can provide boot parameter ip and this will be used to setup
the networking.
Relevant code is in function setup_static_ip
This was supported by debian-live already.
Unfortunately DNS IP is not part of standard IP boot parameter,
http://syslinux.zytor.com/wiki/index.php/SYSLINUX#IPAPPEND_flag_val_.5BPXELINUX_only.5D
so one need pass it separately. User can use dnsip boot time
parameter for providing DNS ip.
In case there is no DNS entry in dhcp output then it is configured statically.
I noticed that you hardcode the dns server IP in your script which is
certainly wrong. There were some issues with getting dns working but I
use IPs only so I would not know.
The do_httpmount part works properly without clashing with run-init.
but do_iscsimount is clashing with run-init.
in do_iscsimount, I am directly mounting the iso image on ${mountpoint}
without extra mount-point ${ext_mountpoint} needed in do_httpmount
Actually mounting an iso is the standard case but mount_httpfs has to
first mount a filesystem containing the iso and create a loop which
requires the additional mountpoint.
Any suggestions on what should be done to avoid clash of do_iscsimount
and run-init will be helpful.
Putting iSCSI outside of the netmount code path because it does not
really belong there.
It just creates a device which should be handled by the standard code path.
Unfortunately the code that searches for iso does not find it on the
iSCSI disk but probing it manually should work.
Attaching a patch that should (mostly) work. Before adding the last
hunk it just mounted the iSCSI disk and exited.
Thanks
Michal
--- usr/share/initramfs-tools/hooks/live.0 2009-09-09 17:33:41.0 +0200
+++ usr/share/initramfs-tools/hooks/live 2009-09-09 17:36:47.0 +0200
@@ -176,3 +176,13 @@
then
copy_exec /usr/bin/curlftpfs /bin
fi
+
+# iSCSI
+if [ -x /usr/bin/iscsiadm ] [ -x /usr/sbin/iscsid ]
+then
+copy_exec /usr/bin/iscsiadm /bin
+copy_exec /usr/sbin/iscsid /bin
+#manual_add_modules ib_iser
+manual_add_modules iscsi_tcp
+manual_add_modules crc32c
+fi
--- usr/share/initramfs-tools/scripts/live.0 2009-09-09 17:33:25.0 +0200
+++ usr/share/initramfs-tools/scripts/live 2009-09-09 17:37:58.0 +0200
@@ -87,6 +87,15 @@
export HTTPFS
;;
+ iscsi=*)
+ISCSI=${ARGUMENT#iscsi=}
+#ip:port - separated by , from target
+ISCSI_PORTAL=${ISCSI%,*}
+#target name
+ISCSI_TARGET=${ISCSI#*,}
+export ISCSI ISCSI_PORTAL ISCSI_TARGET
+;;
+
hostname=*)
HOSTNAME=${ARGUMENT#hostname=}
LIVECONF=changed
@@ -688,7 +697,7 @@
return 0
}
-do_netmount ()
+do_netsetup ()
{
rc=1
@@ -713,6 +722,11 @@
[ -z ${HOSTNAME} ] HOSTNAME=${OLDHOSTNAME}
export HOSTNAME
+ if [ -n ${DEVICE} ]
+ then
+ HWADDR=$(cat /sys/class/net/${DEVICE}/address)
+ fi
+
# Check if we have a network device at all
if ! ls /sys/class/net/eth0 /dev/null 21 \
! ls /sys/class/net/wlan0 /dev/null 21 \
@@ -721,6 +735,11 @@
then
panic No supported network device found, maybe a non-mainline driver is required.
fi
+}
+
+do_netmount()
+{
+ do_netsetup
if [ ${NFSROOT} = auto ]
then
@@ -754,6 +773,24 @@
return ${rc}
}
+do_iscsi()
+{
+ do_netsetup
+ #modprobe ib_iser
+ modprobe iscsi_tcp
+ mkdir -p /etc/iscsi
+ #FIXME this name is supposed to be unique - some date + ifconfig hash?
+ echo InitiatorName=iqn.1993-08.org.debian.live:01:$(echo ${HWADDR} | sed -e s/://g) /etc/iscsi/initiatorname.iscsi
+ iscsid -f
+ iscsiadm --mode discovery --type sendtargets --portal ${ISCSI_PORTAL} -I default
+ iscsiadm --mode node --targetname ${ISCSI_TARGET} --portal ${ISCSI_PORTAL} --login
+ if [ $? != 0 ]
+
Re: Booting Debian-live-501 over HTTP
Hello
2009/9/2 Daniel Baumann dan...@debian.org:
[ CC'ed to ensure you're not missing the answer ;), will follow up on
list only though. ]
Pravin wrote:
I also managed to boot debian over iscsi.
I had to add open-iscsi tools into initramfs for enabling iscsi booting,
but booting over iscsi is 3 times faster than booting over HTTPFS :-)
i would be very, very much interested in patches to live-initramfs to
make iscsi booting out-of-the-box possible, and to live-manual to
document it.
I would be glad if I can help in improving network booting of Debian.
Can you please guide me how and and where should I submit these modifications
for review and testing?
please post it on this list.
I put together a small patch that is supposed to allow booting iso
images from the web.
This is quite helpful as it would obviate the need to store separate
iso images for download and squashfs images for http boot.
Unfortunately, I am not sure it works with the run-init in the current
lenny (or sid) repository.
I use my own (older) patched version which prints out some diagnostic
messages so a failure can be diagnosed easily.
On successful boot the messages just scroll by and are not logged
anywhere, unfortunately.
I did not find any documentation for run-init either. Last time I
debugged it it seemed to leave alone mountpoints and device nodes
directly under / and remove everything else. If there was a mount
point or other non-removable file/directory deeper in the directory
hierarchy it would fail with Directory not empty.
Any testers are welcome, I will try to build a clean image with just
the patch later.
Thanks
Michal
--- usr/share/initramfs-tools/scripts/live.orig 2009-09-03 23:25:10.0 +0200
+++ usr/share/initramfs-tools/scripts/live 2009-09-03 23:26:30.0 +0200
@@ -768,7 +768,7 @@
if [ -n $url ]
then
case ${extension} in
-squashfs|tgz|tar)
+iso|squashfs|tgz|tar)
if [ ${webfile} = FETCH ]
then
case $url in
@@ -797,6 +797,12 @@
fi
[ ${?} -eq 0 ] rc=0
[ ${extension} = tgz ] live_dest=ram
+ if [ ${extension} = iso ]
+ then
+ isoloop=$(setup_loop ${dest}/$(basename ${url}) loop /sys/block/loop* '')
+ mount -t iso9660 ${isoloop} ${mountpoint}
+ rc=${?}
+ fi
break
;;
Re: Booting Debian-live-501 over HTTP
Hi, Thanks for interest :-) i would be very, very much interested in patches to live-initramfs to make iscsi booting out-of-the-box possible, and to live-manual to document it. I have created small patch for live-manual to which will give basic idea about how to use httpfs and iscsifs boot options. I am appending this patch with this email. However I had little problem with live-initramfs code. I could not understand how exactly it works, and the interaction between initramfs-tools and live-initramfs. I could not find any documentation about how I can use live-initramfs code to create custom initramfs for testing purpose. The way I work is by fetching the initramfs image from debian live cd, and modifying it, to support httpfs and iscsi booting. I have created a patch on diff between content of original initramfs and modified initramfs. I am appending that patch also with this email. The binary files that I have added/modified to the initramfs can be downloaded from following URLs. http://rom.etherboot.org/share/pravin/BKO/D/D/DL/httpfs/server/ http://rom.etherboot.org/share/pravin/BKO/D/D/DL/run-init http://rom.etherboot.org/share/pravin/BKO/D/D/DL/modules/ http://rom.etherboot.org/share/pravin/BKO/D/D/DL/iscsi/tools/ This patch is not yet perfect, and contain some code which was added for debugging purpose. I am trying to learn how live-initramfs creates initramfs. Any pointers to documentation and examples of such modification will be very helpful to me. Looking forward for feedback on these patches. Also let me know what should I do next to help in getting these features in debian live? Regards, -- Pravin = == Patch for live-manual = = diff --git a/xml/chapters/customization/bootup.xml b/xml/chapters/customization/bootup.xml index 8daca41..202e395 100644 --- a/xml/chapters/customization/bootup.xml +++ b/xml/chapters/customization/bootup.xml @@ -39,6 +39,7 @@ /para /sect3 + sect3 titleBooting a Debian Live USB/HDD system from a USB stick with Grub/title para @@ -79,6 +80,57 @@ initrd's script in which subdirectory to look for the SquashFS image. /sect3 /sect2 +sect2 id=netboot +titleDiskless booting over Network/title +para +Network booting is specially useful when your system is not having harddisk, or hard disk +is currupted or you do not want to touch it. With netbooting option, user can boot entire +system over network. The only assumption is about network connection on user end. +/para + +para +User need to have small gpxe image or some other way by which user can boot machine over network. +This image can be put on floppy or can be burned on NIC card. This script does the initialization +of hardware, connect to the boot server (eg. http://boot.kernel.org ) and fetches the kernel and +initramfs images. The actual booting is done by this downloaded kernel with help of initramfs image. +/para + +sect3 +titleBooting a Debian Live over HTTP/title +para +If the boot parameter commandhttpfs=URL-to-image/command +is present then http boot is attempted. The URL provided as boot parameter +is used to find out if the image is iso or squashfs. Depending on the +filename extension in the URL. +/para +para +The reason behind providing support for both iso and squashfs is that, +even squashfs netbooting is little faster compared to is iso netbooting. +But iso http booting allows you to give the URL for iso image of any nearby +mirror, resulting much faster booting. +/para +/sect3 + +sect3 +titleBooting a Debian Live over ISCSI/title +If the boot parameter commandiscsifs=URL-to-image/command +is present, then iscsi boot is attempted. The server port for iscsi can +be specifed as part of URL, this allows connections to iscsi servers +running on non-standard ports. ISCSIFS booting also expect one more boot +parameter, namely commandtarget=target-name/command. +The target name specified here is part of target name, which is used to +uniquely identify the target representing debian live iso image when there +are multiple disks hosted. This variable can contain the entire name of target, +or only part of target name which can uniquely identify the debian live iso disk +among all other exported disks. +para +/para +/sect3 + + +/sect2 + + sect2 id=splash titleSplash screens/title para = = == Patch for initramfs = = Binary files initrd_orig/bin/fusermount and initrd_new/bin/fusermount differ Binary files initrd_orig/bin/httpfs and initrd_new/bin/httpfs differ Binary files initrd_orig/bin/run-init and initrd_new/bin/run-init differ diff -rupN initrd_orig/etc/iscsi/initiatorname.iscsi initrd_new/etc/iscsi/initiatorname.iscsi --- initrd_orig/etc/iscsi/initiatorname.iscsi
Re: Booting Debian-live-501 over HTTP
Hi Michal, Sorry that I missed this email in other mails and hence could not update you on my progress. Finally I managed to boot debien live over HTTPFS with ISO. I used unmodified httpfs for this purpose. I also managed to boot debian over iscsi. I had to add open-iscsi tools into initramfs for enabling iscsi booting, but booting over iscsi is 3 times faster than booting over HTTPFS :-) The debian over HTTP is deployed at http://boot.kernel.org And iscsi based debian is deployed on http://rom.etherboot.org/share/pravin/BKO/ You can try them out if you get time. The the modifications that I have done to standard initramfs, and script to generate modified iniramfs are here. http://rom.etherboot.org/share/pravin/BKO/D/D/DL/ and the problems I faced are documented here. http://etherboot.org/wiki/soc/2009/pravin/journal/bkodebian http://etherboot.org/wiki/soc/2009/pravin/journal/bkodebianiscsi The work is still in progress as there are few issues with DNS entries. Also, above deployed servers are in US, so network booting from them is quite slow at other locations. I would be glad if I can help in improving network booting of Debian. Can you please guide me how and and where should I submit these modifications for review and testing? On Wed, Aug 12, 2009 at 9:49 PM, Michal Suchanekhramr...@centrum.cz wrote: Hello, Sorry about the late reply. With this ugly hack, I did managed to get past the error, but I got strange error when gnome GUI started. error There was an error starting the GNOME Settings Daemon. Some things, such as themes, sounds, or background settings may not work correctly. The last error message was: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security blocked the reply, the reply timeout expired, or the network connection was broken GNOME will still try to restart the Settings Daemon next time you log in. I am not running gnome in d-l so I would not know but I expect that gnome works right in the gnome images. I figured out the cause for this warning as delays in booting process. If the remote server is close enough, then this problem does not occur. Also, this warning is little harmless. Users can click on OK and continue with normal working of debian. Thanks Michal I hope that this work will be considered useful. Thank you. -- To UNSUBSCRIBE, email to debian-live-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Booting Debian-live-501 over HTTP
Hello,
Sorry about the late reply.
2009/7/17 Pravin shindepra...@gmail.com:
Hi,
I am working on booting Debian over http, and I faced few problems and
I attempted to solve them.
Following is the approach I took and the the questions I have.
I hope that you will find this work useful.
I have been trying to use httpfs option to boot
debian-live-501-i386-gnome-desktop.iso.
As far as I know iso files are currently not supported by the httpfs method.
Support for these should not be too hard to add, though.
Perhaps the other net methods should support them, too.
But I could not locate the httpfs program in initramfs. So I added
httpfs and fusermount from
http://httpfs.sourceforge.net to /bin of initramfs. With this
I am using httpfs regularly to boot squashfs images but the httpfs
program from sorceforge somehow did not work for me so I use my
modified version. Work is underway on getting this version into
Debian.
modifications debian was able to boot
from filesystem.squashfs hosted on remote webserver.
I also wanted to be able to boot from
debian-live-501-i386-gnome-desktop.iso in addition to
filesystem.squashfs. I attempted this by modifying do_httpmount
to recognize .iso extension.
I mount the iso on ${rootmnt}/iso this iso is then mounted on
${mountpoint}.
Here is the code snippet
code
if [ ${extension} = iso ]
then
modprobe fuse
modprobe loop
mkdir -p ${rootmnt}/iso
httpfs ${url} ${rootmnt}/iso
FILEPATH=`ls ${mountpoint}/iso/*.iso`
mount -t iso9660 $FILEPATH ${mountpoint} -o loop -o ro
return 0
fi
/code
I chose the location ${rootmnt}/iso so that run-init will not try
to delete it in nuking initramfs step.
But still I am getting error
run-init: nuking initramfs contents: Directory not empty
I modified and re-compiled run-init for not ignoring the failure of
nuke_dir(/) and included it in my initramfs.
Unfortunately, the run-init program is a bit of a voodoo and I myself
ended up modifying it to find out what it is doing.
It uses some weird heuristic to find out what should be nuked and what
should be kept.
I use a modified version that prints messages as it recursively nukes
directories so I can tell what went wrong if my system stops at this
point.
With this ugly hack, I did managed to get past the error, but I got
strange error when gnome GUI started.
error
There was an error starting the GNOME Settings Daemon.
Some things, such as themes, sounds, or background settings may not
work correctly.
The last error message was:
Did not receive a reply. Possible causes include: the remote
application did not send a reply, the message bus security blocked the reply,
the reply timeout expired, or the
network connection was broken
GNOME will still try to restart the Settings Daemon next time you log in.
I am not running gnome in d-l so I would not know but I expect that
gnome works right in the gnome images.
Thanks
Michal
--
To UNSUBSCRIBE, email to debian-live-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
