Re: NIC_NL waiting binding to NETLINK_ISCSI socket in /var/log/brcm-iscsi.log
Thanks for help, Eddie. Indeed, we've been running custom version of kernel (forgot to tell that, and it's the only modification to stock installation :) that doesn't have that module included. Here's what I get in log when I start /etc/init.d/iscsi: Jul 30 01:38:30 stage iscsid: iSCSI logger with pid=9177 started! Jul 30 01:38:30 stage iscsid: Missing or Invalid version from /sys/ module/scsi_transport_iscsi/version. Make sure a up to date scsi_transport_iscsi module is loaded and a up todate version of iscsid is running. Exiting... After that, only brcm_iscsiuio is running. So I guess you should follow the iscsid's behaviour and exit if you see the configuration is incorrect/unusable. And log a human readable message, please! Thanks again. On Jul 30, 1:07 am, "Eddie Wai" wrote: > On Fri, 2011-07-29 at 10:43 -0700, Mike Christie wrote: > > Ccing Broadcom. > > > I believe they are working on the log file size issues. They can answer > > the other questions. > > > On 07/29/2011 04:16 AM, Yuri Arabadji wrote: > > > Hi. > > > > During recent log revision, we've found a huge log file present > > > practically on all centos 5.6 hosts. It's located at /var/log/brcm- > > > iscsi.log and being generated by brcm_iscsiuio process. The process > > > writes to it *constantly*, thus generating disk I/O and consuming disk > > > space. By default, centos/rhel's iscsi-initiator-utils rpm doesn't > > > come with logrotate config file, thus the file grows bigger and > > > bigger. Here are the messages in that log file: > > > > ERR [Fri Jul 29 04:11:40 2011]NIC_NL waiting binding to > > > NETLINK_ISCSI socket > > > ERR [Fri Jul 29 04:11:41 2011]NIC_NL waiting binding to > > > NETLINK_ISCSI socket > > > ERR [Fri Jul 29 04:11:42 2011]NIC_NL waiting binding to > > > NETLINK_ISCSI socket > > > > and so on. generated every second (although buffered, thanks for > > > that). I have absolutely no idea what that message means and why it is > > > being produced every second. If there's an error, syslog a fatal > > > message and bail out - that's the proper behaviour. Note, this is an > > > untouched default centos/rhel install. No file modifications have been > > > done to any configs or init scripts. > > > iscsi service is enabled by default on centos/rhel. > > > > Thanks. > > logrotate handling was only added since brcm_iscsiuio-0.7.0.2 > (May/2011), so the inbox version in centos 5.6 would not have this yet. > If you would like an upgrade, you can probably find the tarball for > iscsiuio-0.7.0.4 in the open-iscsi mailing list. > > As for the specific error message, it looks like iscsiuio was not able > to bind the socket opened to get netlink messages. Unfortunately, this > will continue to poll every second until the socket gets binded. I > agree that this is probably not the best way to handle this error. > > The underlying problem appears to be related to a socket communication > problem between the kernel's scsi_transport_iscsi(2) module and > iscsiuio. I have only seen this if the scsi_transport_iscsi(2) module > is not modprobed or when iscsiuio is ran without iscsid. Can you verify > to see if the iscsid process is running and if the > scsi_transport_iscsi(2) module is inserted? Thanks. > > > > > > > > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: NIC_NL waiting binding to NETLINK_ISCSI socket in /var/log/brcm-iscsi.log
On Fri, 2011-07-29 at 10:43 -0700, Mike Christie wrote: > Ccing Broadcom. > > I believe they are working on the log file size issues. They can answer > the other questions. > > On 07/29/2011 04:16 AM, Yuri Arabadji wrote: > > Hi. > > > > During recent log revision, we've found a huge log file present > > practically on all centos 5.6 hosts. It's located at /var/log/brcm- > > iscsi.log and being generated by brcm_iscsiuio process. The process > > writes to it *constantly*, thus generating disk I/O and consuming disk > > space. By default, centos/rhel's iscsi-initiator-utils rpm doesn't > > come with logrotate config file, thus the file grows bigger and > > bigger. Here are the messages in that log file: > > > > ERR [Fri Jul 29 04:11:40 2011]NIC_NL waiting binding to > > NETLINK_ISCSI socket > > ERR [Fri Jul 29 04:11:41 2011]NIC_NL waiting binding to > > NETLINK_ISCSI socket > > ERR [Fri Jul 29 04:11:42 2011]NIC_NL waiting binding to > > NETLINK_ISCSI socket > > > > and so on. generated every second (although buffered, thanks for > > that). I have absolutely no idea what that message means and why it is > > being produced every second. If there's an error, syslog a fatal > > message and bail out - that's the proper behaviour. Note, this is an > > untouched default centos/rhel install. No file modifications have been > > done to any configs or init scripts. > > iscsi service is enabled by default on centos/rhel. > > > > Thanks. > > logrotate handling was only added since brcm_iscsiuio-0.7.0.2 (May/2011), so the inbox version in centos 5.6 would not have this yet. If you would like an upgrade, you can probably find the tarball for iscsiuio-0.7.0.4 in the open-iscsi mailing list. As for the specific error message, it looks like iscsiuio was not able to bind the socket opened to get netlink messages. Unfortunately, this will continue to poll every second until the socket gets binded. I agree that this is probably not the best way to handle this error. The underlying problem appears to be related to a socket communication problem between the kernel's scsi_transport_iscsi(2) module and iscsiuio. I have only seen this if the scsi_transport_iscsi(2) module is not modprobed or when iscsiuio is ran without iscsid. Can you verify to see if the iscsid process is running and if the scsi_transport_iscsi(2) module is inserted? Thanks. > > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: NIC_NL waiting binding to NETLINK_ISCSI socket in /var/log/brcm-iscsi.log
Ccing Broadcom. I believe they are working on the log file size issues. They can answer the other questions. On 07/29/2011 04:16 AM, Yuri Arabadji wrote: > Hi. > > During recent log revision, we've found a huge log file present > practically on all centos 5.6 hosts. It's located at /var/log/brcm- > iscsi.log and being generated by brcm_iscsiuio process. The process > writes to it *constantly*, thus generating disk I/O and consuming disk > space. By default, centos/rhel's iscsi-initiator-utils rpm doesn't > come with logrotate config file, thus the file grows bigger and > bigger. Here are the messages in that log file: > > ERR [Fri Jul 29 04:11:40 2011]NIC_NL waiting binding to > NETLINK_ISCSI socket > ERR [Fri Jul 29 04:11:41 2011]NIC_NL waiting binding to > NETLINK_ISCSI socket > ERR [Fri Jul 29 04:11:42 2011]NIC_NL waiting binding to > NETLINK_ISCSI socket > > and so on. generated every second (although buffered, thanks for > that). I have absolutely no idea what that message means and why it is > being produced every second. If there's an error, syslog a fatal > message and bail out - that's the proper behaviour. Note, this is an > untouched default centos/rhel install. No file modifications have been > done to any configs or init scripts. > iscsi service is enabled by default on centos/rhel. > > Thanks. > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
