Re: [Oscar-users] Replace bnx2.ko module into initrd.img
Hi DongInn, This is to let you know that I fixed my issue with Broadcom Netxtreme II card and I successfully installed my node. If this could be of any help to others.. Problem as I see it (after dozens of tests): The bnx2 driver v. 2.2.1 used onto the master node (came with CentOS 6.3) requests .fw files in /lib/firmware. The.fw files was missing from the initrd.img and the SIS oscarimage. The /lib/firmware files were not included into the SIS oscarimage and therefore not into the initrd.img. After adding them to the initrd.img it was impossible to make the kernel load the .fw files during the PXE boot. Solution: I downloaded the last driver archive Netxtreme2-linux-7.4.27.tar.gz from Broadcom and compiled the drivers. I loaded my new module to check it works onto the master node (same hardware). I copied the bnx2.ko file to the initrd.img UYOK (uncompressed archive) and recreated the image. NOTE: The new bnx2.ko does not depend on any .fw files. I booted PXE and it worked. Thank you for your support. Best Regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
Re: [Oscar-users] Replace bnx2.ko module into initrd.img
Hi Costel, This is great! Thanks for sharing your solution. Regards, -- - DongInn On Mar 12, 2013, at 4:43 AM, Costel Seitan csei...@slb.com wrote: Hi DongInn, This is to let you know that I fixed my issue with Broadcom Netxtreme II card and I successfully installed my node. If this could be of any help to others.. Problem as I see it (after dozens of tests): The bnx2 driver v. 2.2.1 used onto the master node (came with CentOS 6.3) requests .fw files in /lib/firmware. The.fw files was missing from the initrd.img and the SIS oscarimage. The /lib/firmware files were not included into the SIS oscarimage and therefore not into the initrd.img. After adding them to the initrd.img it was impossible to make the kernel load the .fw files during the PXE boot. Solution: I downloaded the last driver archive Netxtreme2-linux-7.4.27.tar.gz from Broadcom and compiled the drivers. I loaded my new module to check it works onto the master node (same hardware). I copied the bnx2.ko file to the initrd.img UYOK (uncompressed archive) and recreated the image. NOTE: The new bnx2.ko does not depend on any .fw files. I booted PXE and it worked. Thank you for your support. Best Regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
Re: [Oscar-users] Replace bnx2.ko module into initrd.img
Hi DongInn, Just to clarify things. I loaded the modules onto my master server which was installed with CentOS 6.3 media and has the same hardware (the same Broadcom network card, same disks etc. ). All the modules are loaded or load correctly. The node (that I try to install): 1. It boots PXE correctly 2. It retrieves and loads the boel kernel (bzimage) 3. When trying to load the initrd.img it fails not anymore at loading the NIC module (bnx2) but when trying to mount /dev/ram (as specified into the default file retrieved from the pxe server) NOTE: It stops before doing DHCPDISCOVER and so before retrieving oscarimge through rsync From the messages I see at the console, the new bnx2 module seems to be found and loaded correctly as I succeeded to include the right driver and .fw files into the initrd.image. The initrd.image was created with something like dracut /tftpboot/initramfs-` uname -r`.img `uname -r` with line add_drivers+= ...bnx2 tg3 megaraid_sas .. into /etc/dracut.conf. I checked afterwards and all the specified modules are inside the initrd.img. However, when comparing this last image with the previous ones I see that some tools are not included like /sbin/mkfs.cramfs for instance. I tried to include missing tools with dracut --include /sbin/mkfs.cramfs .. or dracut --install mkfs.cramfs .. NOTE: I tried the different options of dracut tool like for instance to generate a initrd.img for the running kernel onto the master (after I checked that all of my modules were loaded). It did rather generate and incomplete and unusable initrd.img. I believe the best way would be to find the command that has been used when creating the UYOK kernel and initrd.img. Kind Regards, Costel From: DongInn Kim [mailto:di...@cs.indiana.edu] Sent: Friday, March 08, 2013 2:32 PM To: oscar-users@lists.sourceforge.net Subject: Re: [Oscar-users] Replace bnx2.ko module into initrd.img Costel, What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Well, this should be possible. That's why we use OSCAR. OSCAR is supposed to be able to host heterogenous cluster. Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? I have setup an OSCAR cluster with bnx2x network driver and its firmware on ubuntu box Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? It is not really recommended but I think it is OK to update oscarimage directly as long as you know exactly what you are doing. BTW, I would like to clarify your problem. Can your second machine (e.g., 2nd M) do pxeboot properly? - Can 2nd M load up the bnx2 module and get the network connection? - Did it get the oscarimge through rsync or whatever you choose? How did you get the bnx2 module and its firmware? Have you tried to load up bnx2 module on your head node and does it work? Regards, -- - DongInn On Mar 8, 2013, at 7:52 AM, Costel Seitan csei...@slb.commailto:csei...@slb.com wrote: Hi DongInn, In fact I only have 1 oscarimage and what I did till now was to modify the modules already available into the initrd.img but not loaded during the boot phase. So I modified initrd.img generated by systemimager with UYOK selected. This worked with the first node. This is what I tried to do with the second node (but different hardware), using the same kernel but with a modified initrd.file file. The problem is that the bnx2.ko into the initrd.img is not the right one and it does not contains all the requested .fw files. I copied the module .fw files present onto the master node but I got the errors about the signature key. What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? I know there are a lot of questions for one posted message. :-) Kind Regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Oscar-users mailing list Oscar-users@lists.sourceforge.netmailto:Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
Re: [Oscar-users] Replace bnx2.ko module into initrd.img
Hi DongInn, In fact I only have 1 oscarimage and what I did till now was to modify the modules already available into the initrd.img but not loaded during the boot phase. So I modified initrd.img generated by systemimager with UYOK selected. This worked with the first node. This is what I tried to do with the second node (but different hardware), using the same kernel but with a modified initrd.file file. The problem is that the bnx2.ko into the initrd.img is not the right one and it does not contains all the requested .fw files. I copied the module .fw files present onto the master node but I got the errors about the signature key. What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? I know there are a lot of questions for one posted message. :-) Kind Regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
Re: [Oscar-users] Replace bnx2.ko module into initrd.img
Costel, What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Well, this should be possible. That's why we use OSCAR. OSCAR is supposed to be able to host heterogenous cluster. Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? I have setup an OSCAR cluster with bnx2x network driver and its firmware on ubuntu box Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? It is not really recommended but I think it is OK to update oscarimage directly as long as you know exactly what you are doing. BTW, I would like to clarify your problem. Can your second machine (e.g., 2nd M) do pxeboot properly? - Can 2nd M load up the bnx2 module and get the network connection? - Did it get the oscarimge through rsync or whatever you choose? How did you get the bnx2 module and its firmware? Have you tried to load up bnx2 module on your head node and does it work? Regards, -- - DongInn On Mar 8, 2013, at 7:52 AM, Costel Seitan csei...@slb.com wrote: Hi DongInn, In fact I only have 1 oscarimage and what I did till now was to modify the modules already available into the initrd.img but not loaded during the boot phase. So I modified initrd.img generated by systemimager with UYOK selected. This worked with the first node. This is what I tried to do with the second node (but different hardware), using the same kernel but with a modified initrd.file file. The problem is that the bnx2.ko into the initrd.img is not the right one and it does not contains all the requested .fw files. I copied the module .fw files present onto the master node but I got the errors about the signature key. What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? I know there are a lot of questions for one posted message. :-) Kind Regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
Re: [Oscar-users] Replace bnx2.ko module into initrd.img
Have you tried to load up bnx2 module on your head node and does it work? What I meant by does it work is that the kernel module could be loaded up successfully. It does not mean that you can have the network connection via that kernel module because I assume that your head node does not have the broadcom network card. I hope that it is clear. Regards, -- - DongInn On Mar 8, 2013, at 8:32 AM, DongInn Kim di...@cs.indiana.edu wrote: Costel, What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Well, this should be possible. That's why we use OSCAR. OSCAR is supposed to be able to host heterogenous cluster. Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? I have setup an OSCAR cluster with bnx2x network driver and its firmware on ubuntu box Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? It is not really recommended but I think it is OK to update oscarimage directly as long as you know exactly what you are doing. BTW, I would like to clarify your problem. Can your second machine (e.g., 2nd M) do pxeboot properly? - Can 2nd M load up the bnx2 module and get the network connection? - Did it get the oscarimge through rsync or whatever you choose? How did you get the bnx2 module and its firmware? Have you tried to load up bnx2 module on your head node and does it work? Regards, -- - DongInn On Mar 8, 2013, at 7:52 AM, Costel Seitan csei...@slb.com wrote: Hi DongInn, In fact I only have 1 oscarimage and what I did till now was to modify the modules already available into the initrd.img but not loaded during the boot phase. So I modified initrd.img generated by systemimager with UYOK selected. This worked with the first node. This is what I tried to do with the second node (but different hardware), using the same kernel but with a modified initrd.file file. The problem is that the bnx2.ko into the initrd.img is not the right one and it does not contains all the requested .fw files. I copied the module .fw files present onto the master node but I got the errors about the signature key. What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? I know there are a lot of questions for one posted message. :-) Kind Regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
Re: [Oscar-users] Replace bnx2.ko module into initrd.img
DongInn, Just to let you know I succeeded in creating a new initrd.img with my modules inside. I used dracut tool with a long list of modules in /etc/dracut.conf and I included mine inside. I noticed that it successfully loaded the bnx2 module and the .fw files. Anyway, I still continue to have troubles because it generates a kernel panic just afterwards (can't find block:/dev/ram ). I suppose it is connected to the filesystems supported. I will continue to troubleshoot this. Kind Regards, Costel -Original Message- From: Costel Seitan [mailto:csei...@slb.com] Sent: Friday, March 08, 2013 1:53 PM To: oscar-users@lists.sourceforge.net Subject: Re: [Oscar-users] Replace bnx2.ko module into initrd.img Hi DongInn, In fact I only have 1 oscarimage and what I did till now was to modify the modules already available into the initrd.img but not loaded during the boot phase. So I modified initrd.img generated by systemimager with UYOK selected. This worked with the first node. This is what I tried to do with the second node (but different hardware), using the same kernel but with a modified initrd.file file. The problem is that the bnx2.ko into the initrd.img is not the right one and it does not contains all the requested .fw files. I copied the module .fw files present onto the master node but I got the errors about the signature key. What I would like to achieve is to have a single oscarimage and a single BOEL kernel and initrd.img files that contains all the modules needed by my 3 types of hardware. Do you believe this would be possible? Do you have experience in replacing the NIC (bnx2.ko) module with an updated one? Can I force the creation of kernel and initrd.img files to include (really) all the available drivers into my oscarimage? I know there are a lot of questions for one posted message. :-) Kind Regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
Re: [Oscar-users] Replace bnx2.ko module into initrd.img
Hi Costel, So, do you have two oscar images for your two different kinds of client nodes? Maybe you can combined them as long as your oscarimage has all the necessary kernel modules and /etc/modprobe.conf is properly configured, I think it would be fine. But the initrd should be differently setup to load up the corresponding network module and firmware. To fix the issue above I tried to include the files bnx2.ko and .fw found into my oscarimage but I keep receiving messages like “Module signed with unknown public key” and “Can’t insert ‘lib/modules/../bnx2.ko’: Required key not available.” I would like to check the followings: - Any kernel dependencies are there? (i.e., any required kernel module before loading bnx2) - Make sure that bnx2.ko and its firmware are loaded properly on any actually running machine Regards, -- - DongInn On Mar 7, 2013, at 1:47 PM, Costel Seitan csei...@slb.com wrote: All, I am back with good news but also a new issue. I successfully installed my first node using OSCAR 6 on CentOS 6. In order to do this I had to load 2 drivers (tg3 and megaraid_sas) into the /tftpboot/initrd.img to deal with the new hardware. Now I am trying to install older nodes which use Broadcom Netxtreme II driver (bnx2) to deal with the installed NICs. The installation is failing because of a missing file into the initrd.img: bnx2-mips-06-6.2.3.fw. The module bnx2.ko into the initrd.img generated by OSCAR seems to differ from the one installed onto the oscarimage and the one installed onto the system. md5sum or modinfo shows that the one in oscarimage and the one installed onto the master node are the same. I also found the bnx2-mips-06-6.2.3.fw into the system and oscarimg. To fix the issue above I tried to include the files bnx2.ko and .fw found into my oscarimage but I keep receiving messages like “Module signed with unknown public key” and “Can’t insert ‘lib/modules/../bnx2.ko’: Required key not available.” I tried to remove the signature but I did not succeed. It looks like I need to place a key into the kernel file. Does anybody know how to work around this issue? Thank you in advance, Best regards, Costel SEITAN -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users
