Re: iSCSI boot mussings
On 16.03.2007, at 16:59, John Nielsen wrote: A truly standalone iSCSI client will most likely want to use a TOE card, which to the OS looks like any other SCSI adapter. (I'm unsure which if any such cards are currently supported in FreeBSD, but that's a tangential question.) Maybe someone feels like playing with these: http://www.myri.com/ Myri-10G/10gbe_solutions.html Achim as far as I can tell, this is a 10G card, without any iSCSI support. danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: iSCSI boot mussings
... A truly standalone iSCSI client will most likely want to use a TOE card, which to the OS looks like any other SCSI adapter. (I'm unsure which if any such cards are currently supported in FreeBSD, but that's a tangential question.) I really doubt this. TOE iSCSI cards are quite rare. Qlogic has one, and it could work under FreeBSD, as it looks similar the other Qlogic HBA cards. But they are still pretty rare. And TOE doesn't necessarily imply iSCSI. Various cards existing ethernet cards have various levels of offload support now. I think the general case is going to succeed: ethernet cards with acceleration, not iSCSI HBAs. Machines with iSCSI-capable BIOS'es are an inbetween case. Allowing such machines to be standalone clients would require things like the initiator name, the initiator's IP address and netmask, the target's IP address, and the target's (volume) name to be hard-coded in the kernel. It would be nice to support this scenario, but IMO it's the one with the lowest benefit/cost ratio. Maybe, maybe not. Probably more people have iSCSI BIOSes in their servers now, than iSCSI dedicated HBAs. IBM introduced iSCSI boot code into many of their server BIOSes last year. I don't think it is that hard either. I don't think you will have to hard code into the kernel, but at least make sure the targets match what the BIOS has. It could just be a boot option, as the boot loader just uses BIOS calls, which would hit read from iSCSI. If you don't have an iSCSI BIOS in your server (which probably every Dell, IBM, and HP server will have standard in every server this year), and you don't have one of the very rare iSCSI dedicated HBAs, then PXE booting an iSCSI enabled kernel is probably the way to go. Making it easy to integrate iSCSI into existing environments (diskless or not) is IMO the biggest hole in the current implementation (the missing rc and fstab bits I mentioned before). JN Tom ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
iSCSI boot mussings
Hi, Now that I have my hands on a server that can boot iSCSI, I started to look into it. After figuring out what magic is needed in the dhcpd.conf (just add option root-path iscsi:target-iptarget-name) I can boot FreeBSD to the point that it can't find a root device, and assuming that some more magic can be applied (ala NFS), I'm just wondering aloud, if it's realy worth the efford. For a PXE based diskless solution, you need 1 - a working dhcpd 2 - a working tftpd 3 - a working NFS server with the exported root fs. appplying some minor magic, you can have only one read-only fs. For an iSCSI based diskless solution, you need 1- a working dhcpd 2- a working iscsi-initiator, unless the BIOS can be used. 3- a working target with a root fs (one for each client, unless applying 3 from the above). Hybrid solution: boot via PXE, but mount root via iSCSI So, what say you all? danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: iSCSI boot mussings
On Friday 16 March 2007 07:34, Danny Braniss wrote: Hi, Now that I have my hands on a server that can boot iSCSI, I started to look into it. After figuring out what magic is needed in the dhcpd.conf (just add option root-path iscsi:target-iptarget-name) I can boot FreeBSD to the point that it can't find a root device, and assuming that some more magic can be applied (ala NFS), I'm just wondering aloud, if it's realy worth the efford. For a PXE based diskless solution, you need 1 - a working dhcpd 2 - a working tftpd 3 - a working NFS server with the exported root fs. appplying some minor magic, you can have only one read-only fs. For an iSCSI based diskless solution, you need 1- a working dhcpd 2- a working iscsi-initiator, unless the BIOS can be used. 3- a working target with a root fs (one for each client, unless applying 3 from the above). Hybrid solution: boot via PXE, but mount root via iSCSI So, what say you all? From the kernel's perspective (at the moment just prior to mounting root), is there a difference between the last two approaches? The situation as I see it (in both cases) is that the kernel is loaded into memory (by some magical means which is at this point irrelevant), and now has to locate a root device using only what it already has to bootstrap the process. If what it already has includes BOOTP code then it's possible to get some additional information externally. Whether the initial magic was PXE or BIOS-based iSCSI, the kernel has to have its own storage drivers and do its own network setup, right? I think there are some benefits to being able to do this, but perhaps 90% of them could be realized with what we already have (iscontrol and the iscsi_initiator kernel module) plus some rc and fstab glue. Any kind of diskless server farm needs at least one master server to run dhcpd, and if you have it doing that you might as well have it do NFS and tftpd as well. Pretty much any client you'd want to use as an iSCSI initiator will have a decent NIC which nowadays implies PXE support. So it shouldn't be hard to get root mounted using currently available means (local disk or PXE+NFS), and from there have the option of using iSCSI for other partitions (including /usr). A truly standalone iSCSI client will most likely want to use a TOE card, which to the OS looks like any other SCSI adapter. (I'm unsure which if any such cards are currently supported in FreeBSD, but that's a tangential question.) Machines with iSCSI-capable BIOS'es are an inbetween case. Allowing such machines to be standalone clients would require things like the initiator name, the initiator's IP address and netmask, the target's IP address, and the target's (volume) name to be hard-coded in the kernel. It would be nice to support this scenario, but IMO it's the one with the lowest benefit/cost ratio. Making it easy to integrate iSCSI into existing environments (diskless or not) is IMO the biggest hole in the current implementation (the missing rc and fstab bits I mentioned before). JN ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: iSCSI boot mussings
Hi, Now that I have my hands on a server that can boot iSCSI, I started to look into it. After figuring out what magic is needed in the dhcpd.conf (just add option root-path iscsi:target-iptarget-name) I can boot FreeBSD to the point that it can't find a root device, and assuming that some more magic can be applied (ala NFS), I'm just wondering aloud, if it's realy worth the efford. For a PXE based diskless solution, you need 1 - a working dhcpd 2 - a working tftpd 3 - a working NFS server with the exported root fs. appplying some minor magic, you can have only one read-only fs. For an iSCSI based diskless solution, you need 1- a working dhcpd 2- a working iscsi-initiator, unless the BIOS can be used. 3- a working target with a root fs (one for each client, unless applying 3 from the above). Hybrid solution: boot via PXE, but mount root via iSCSI So, what say you all? From the kernel's perspective (at the moment just prior to mounting root), is there a difference between the last two approaches? with PXE, all the information, including the nfs-file-handle used to access the root partition is passed to the kernel (pxeboot), and nfsdiskless.c does the rest. With iSCSI zero, will need to be done. The situation as I see it (in both cases) is that the kernel is loaded into memory (by some magical means which is at this point irrelevant), and now has to locate a root device using only what it already has to bootstrap the process. If what it already has includes BOOTP code then it's possible to get some additional information externally. with PXE this is done via pxeboot (and my modified libstand/bootp.c which copies the dhcp obtained tags to the kernel environment). Whether the initial magic was PXE or BIOS-based iSCSI, the kernel has to have its own storage drivers and do its own network setup, right? true, since FreeBSD does not like to use BIOS provided 'drivers'. the problem is 'discovering' the device used to boot, and here there is an added complication, the iSCSI is on top of the network card, so initializing the NIC will probably disconnect the iSCSI connection. It would be nice to use the BIOS supplied 'glue' to mount root, so that it can read /boot/loader.conf etc, then once the kernel is ready, remount root, I think there are some benefits to being able to do this, but perhaps 90% of them could be realized with what we already have (iscontrol and the iscsi_initiator kernel module) plus some rc and fstab glue. but this can happen only after the kernel is running. horse/cart ... Any kind of diskless server farm needs at least one master server to run dhcpd, and if you have it doing that you might as well have it do NFS and tftpd as well. Pretty much any client you'd want to use as an iSCSI initiator will have a decent NIC which nowadays implies PXE support. So it shouldn't be hard to get root mounted using currently available means (local disk or PXE+NFS), and from there have the option of using iSCSI for other partitions (including /usr). I think that PXE is already concidered 'legacy' :-) A truly standalone iSCSI client will most likely want to use a TOE card, which to the OS looks like any other SCSI adapter. (I'm unsure which if any such cards are currently supported in FreeBSD, but that's a tangential question.) This was my thinking too, but the few TOEs I've seen do not emulate a scsi device! and they are not cheap (last i checked they were over 1000$). Machines with iSCSI-capable BIOS'es are an inbetween case. Allowing such machines to be standalone clients would require things like the initiator name, the initiator's IP address and netmask, the target's IP address, and the target's (volume) name to be hard-coded in the kernel. It would be nice to support this scenario, but IMO it's the one with the lowest benefit/cost ratio. and some more, if you want password/CHAP ... Making it easy to integrate iSCSI into existing environments (diskless or not) is IMO the biggest hole in the current implementation (the missing rc and fstab bits I mentioned before). yes, but this is not a real problem, I have an almost working script. JN ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: iSCSI boot mussings
On 16.03.2007, at 16:59, John Nielsen wrote: A truly standalone iSCSI client will most likely want to use a TOE card, which to the OS looks like any other SCSI adapter. (I'm unsure which if any such cards are currently supported in FreeBSD, but that's a tangential question.) Maybe someone feels like playing with these: http://www.myri.com/ Myri-10G/10gbe_solutions.html Achim ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: iSCSI boot mussings
On 3/16/07, Achim Patzner [EMAIL PROTECTED] wrote: On 16.03.2007, at 16:59, John Nielsen wrote: A truly standalone iSCSI client will most likely want to use a TOE card, which to the OS looks like any other SCSI adapter. (I'm unsure which if any such cards are currently supported in FreeBSD, but that's a tangential question.) Maybe someone feels like playing with these: http://www.myri.com/ Myri-10G/10gbe_solutions.html There are at least a half dozen if not a dozen HBA iSCSI initiators. The only gating factor is price. A pxebooted ramdisk image with an iSCSI software initiator in it is probably the best way to go for most people. I don't think that there is a FreeBSD driver for any of the current hardware iSCSI HBAs. -Kip ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
