Re: [Freeipmi-users] getting internal IPMI error
When FreeIPMI is doing it's communication w/ the BMC there's an error pretty deep down causing grief. Do you know what inband driver you are using? Is it posssible you have the situation described here? http://www.gnu.org/software/freeipmi/freeipmi-faq.html#Why-am-I-seeing-so-many-_0027internal-IPMI-error_0027-or-_0027driver-busy_0027-messages_003f If not, we'd have to dig a little deeper. Any chance you and download the tar.gz off the website and build with ./configure --enable-debug --enable-trace which can give us way more info. Al On Wed, 2015-03-11 at 09:37 -0700, and...@thepw.com wrote: Hi Al, Thank you for your help. It is still happening a lot and only on this one server, the other identical server does not have this problem. These servers are idle right now, they are not being used but will be for databases. I ran the Nagios command manually and get the error after a few tries. I ran the freeipmi command that Nagios is running manually and got the error after a few tries. I turned on debug and after a few verbose outputs, I get this, its rather terse: # sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --debug = Get SDR Repository Info Request = KCS Header: [ 0h] = lun[ 2b] [ Ah] = net_fn[ 6b] IPMI Command Data: -- [ 20h] = cmd[ 8b] ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error # Do you know what could cause this? Thanks, Andrew Original Message Subject: Re: [Freeipmi-users] getting internal IPMI error From: Albert Chu ch...@llnl.gov Date: Tue, March 10, 2015 1:32 pm To: and...@thepw.com Cc: freeipmi-users@gnu.org Hi Andrew, If it's sporadic, I'm not that concerned, as it could be due to a busy system. If it happens very frequently, --debug output would be helpful. Although that'll lead to a lot of log output for Nagios. Al On Tue, 2015-03-10 at 08:14 -0700, and...@thepw.com wrote: Hi, I've setup Nagios to monitor IPMI on our 2 new Supermicro servers. The one is working find, the other sporadically but frequently is returning some errors. The page at https://www.gnu.org/software/freeipmi/manpages/man8/ipmi-sel.8.html suggested emailing here. Please let me know if you can help or if you need any more information. Thanks, Andrew Errors reported to Nagios: [03-10-2015 03:28:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ID [03-10-2015 08:55:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;HARD;2;ID Service Unknown[03-10-2015 08:54:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_cmd_get_device_id: bad completion code [03-10-2015 09:00:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;OK;HARD;2;IPMI Status: OK [03-09-2015 21:02:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sel_parse: internal IPMI error [03-09-2015 19:11:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 19:00:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 18:34:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sensor_read: internal IPMI error from website above: internal IPMI error - An IPMI error has occurred that FreeIPMI does not know how to handle. Please e-mail freeipmi-users@gnu.org to report the issue. ___ Freeipmi-users mailing list Freeipmi-users@gnu.org https://lists.gnu.org/mailman/listinfo/freeipmi-users -- Albert Chu ch...@llnl.gov Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -- Albert Chu ch...@llnl.gov Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory ___ Freeipmi-users mailing list Freeipmi-users@gnu.org https://lists.gnu.org/mailman/listinfo/freeipmi-users
Re: [Freeipmi-users] getting internal IPMI error
Hey, that might be it. Although the servers are identical. I built them at different times so they have different versions of CentOS: cat /etc/*release* CentOS release 6.6 (Final) CentOS release 6.6 (Final) CentOS release 6.6 (Final) cpe:/o:centos:linux:6:GA cat /etc/*release* CentOS release 6.4 (Final) CentOS release 6.4 (Final) CentOS release 6.4 (Final) cpe:/o:centos:linux:6:GA I could upgrade the older system to fix it. Would it still be helpful for you if I download and install it as you asked or should I just update? Thanks, Andrew Original Message Subject: RE: [Freeipmi-users] getting internal IPMI error From: Albert Chu ch...@llnl.gov Date: Wed, March 11, 2015 1:26 pm To: and...@thepw.com Cc: freeipmi-users@gnu.org When FreeIPMI is doing it's communication w/ the BMC there's an error pretty deep down causing grief. Do you know what inband driver you are using? Is it posssible you have the situation described here? http://www.gnu.org/software/freeipmi/freeipmi-faq.html#Why-am-I-seeing-so-many-_0027internal-IPMI-error_0027-or-_0027driver-busy_0027-messages_003f If not, we'd have to dig a little deeper. Any chance you and download the tar.gz off the website and build with ./configure --enable-debug --enable-trace which can give us way more info. Al On Wed, 2015-03-11 at 09:37 -0700, and...@thepw.com wrote: Hi Al, Thank you for your help. It is still happening a lot and only on this one server, the other identical server does not have this problem. These servers are idle right now, they are not being used but will be for databases. I ran the Nagios command manually and get the error after a few tries. I ran the freeipmi command that Nagios is running manually and got the error after a few tries. I turned on debug and after a few verbose outputs, I get this, its rather terse: # sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --debug = Get SDR Repository Info Request = KCS Header: [ 0h] = lun[ 2b] [ Ah] = net_fn[ 6b] IPMI Command Data: -- [ 20h] = cmd[ 8b] ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error # Do you know what could cause this? Thanks, Andrew Original Message Subject: Re: [Freeipmi-users] getting internal IPMI error From: Albert Chu ch...@llnl.gov Date: Tue, March 10, 2015 1:32 pm To: and...@thepw.com Cc: freeipmi-users@gnu.org Hi Andrew, If it's sporadic, I'm not that concerned, as it could be due to a busy system. If it happens very frequently, --debug output would be helpful. Although that'll lead to a lot of log output for Nagios. Al On Tue, 2015-03-10 at 08:14 -0700, and...@thepw.com wrote: Hi, I've setup Nagios to monitor IPMI on our 2 new Supermicro servers. The one is working find, the other sporadically but frequently is returning some errors. The page at https://www.gnu.org/software/freeipmi/manpages/man8/ipmi-sel.8.html suggested emailing here. Please let me know if you can help or if you need any more information. Thanks, Andrew Errors reported to Nagios: [03-10-2015 03:28:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ID [03-10-2015 08:55:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;HARD;2;ID Service Unknown[03-10-2015 08:54:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_cmd_get_device_id: bad completion code [03-10-2015 09:00:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;OK;HARD;2;IPMI Status: OK [03-09-2015 21:02:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sel_parse: internal IPMI error [03-09-2015 19:11:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 19:00:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 18:34:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sensor_read: internal IPMI error from website above: internal IPMI error - An IPMI error has occurred that FreeIPMI does not know how to handle. Please e-mail freeipmi-users@gnu.org to report the issue. ___ Freeipmi-users mailing list Freeipmi-users@gnu.org https://lists.gnu.org/mailman/listinfo/freeipmi-users -- Albert
Re: [Freeipmi-users] getting internal IPMI error
I'd just update. This is a well known issue. I know that Redhat eventually backtracked on that change due to complaints. Al On Wed, 2015-03-11 at 11:14 -0700, and...@thepw.com wrote: Hey, that might be it. Although the servers are identical. I built them at different times so they have different versions of CentOS: cat /etc/*release* CentOS release 6.6 (Final) CentOS release 6.6 (Final) CentOS release 6.6 (Final) cpe:/o:centos:linux:6:GA cat /etc/*release* CentOS release 6.4 (Final) CentOS release 6.4 (Final) CentOS release 6.4 (Final) cpe:/o:centos:linux:6:GA I could upgrade the older system to fix it. Would it still be helpful for you if I download and install it as you asked or should I just update? Thanks, Andrew Original Message Subject: RE: [Freeipmi-users] getting internal IPMI error From: Albert Chu ch...@llnl.gov Date: Wed, March 11, 2015 1:26 pm To: and...@thepw.com Cc: freeipmi-users@gnu.org When FreeIPMI is doing it's communication w/ the BMC there's an error pretty deep down causing grief. Do you know what inband driver you are using? Is it posssible you have the situation described here? http://www.gnu.org/software/freeipmi/freeipmi-faq.html#Why-am-I-seeing-so-many-_0027internal-IPMI-error_0027-or-_0027driver-busy_0027-messages_003f If not, we'd have to dig a little deeper. Any chance you and download the tar.gz off the website and build with ./configure --enable-debug --enable-trace which can give us way more info. Al On Wed, 2015-03-11 at 09:37 -0700, and...@thepw.com wrote: Hi Al, Thank you for your help. It is still happening a lot and only on this one server, the other identical server does not have this problem. These servers are idle right now, they are not being used but will be for databases. I ran the Nagios command manually and get the error after a few tries. I ran the freeipmi command that Nagios is running manually and got the error after a few tries. I turned on debug and after a few verbose outputs, I get this, its rather terse: # sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --debug = Get SDR Repository Info Request = KCS Header: [ 0h] = lun[ 2b] [ Ah] = net_fn[ 6b] IPMI Command Data: -- [ 20h] = cmd[ 8b] ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error # Do you know what could cause this? Thanks, Andrew Original Message Subject: Re: [Freeipmi-users] getting internal IPMI error From: Albert Chu ch...@llnl.gov Date: Tue, March 10, 2015 1:32 pm To: and...@thepw.com Cc: freeipmi-users@gnu.org Hi Andrew, If it's sporadic, I'm not that concerned, as it could be due to a busy system. If it happens very frequently, --debug output would be helpful. Although that'll lead to a lot of log output for Nagios. Al On Tue, 2015-03-10 at 08:14 -0700, and...@thepw.com wrote: Hi, I've setup Nagios to monitor IPMI on our 2 new Supermicro servers. The one is working find, the other sporadically but frequently is returning some errors. The page at https://www.gnu.org/software/freeipmi/manpages/man8/ipmi-sel.8.html suggested emailing here. Please let me know if you can help or if you need any more information. Thanks, Andrew Errors reported to Nagios: [03-10-2015 03:28:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ID [03-10-2015 08:55:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;HARD;2;ID Service Unknown[03-10-2015 08:54:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_cmd_get_device_id: bad completion code [03-10-2015 09:00:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;OK;HARD;2;IPMI Status: OK [03-09-2015 21:02:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sel_parse: internal IPMI error [03-09-2015 19:11:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 19:00:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 18:34:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sensor_read: internal IPMI error from website above
Re: [Freeipmi-users] getting internal IPMI error
Hi Andrew, If it's sporadic, I'm not that concerned, as it could be due to a busy system. If it happens very frequently, --debug output would be helpful. Although that'll lead to a lot of log output for Nagios. Al On Tue, 2015-03-10 at 08:14 -0700, and...@thepw.com wrote: Hi, I've setup Nagios to monitor IPMI on our 2 new Supermicro servers. The one is working find, the other sporadically but frequently is returning some errors. The page at https://www.gnu.org/software/freeipmi/manpages/man8/ipmi-sel.8.html suggested emailing here. Please let me know if you can help or if you need any more information. Thanks, Andrew Errors reported to Nagios: [03-10-2015 03:28:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ID [03-10-2015 08:55:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;HARD;2;ID Service Unknown[03-10-2015 08:54:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_cmd_get_device_id: bad completion code [03-10-2015 09:00:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;OK;HARD;2;IPMI Status: OK [03-09-2015 21:02:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sel_parse: internal IPMI error [03-09-2015 19:11:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 19:00:57] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-databaseServer8.localhost: internal IPMI error [03-09-2015 18:34:47] SERVICE ALERT: databaseServer8;IPMI SENSORS;UNKNOWN;SOFT;1;ipmi_sensor_read: internal IPMI error from website above: internal IPMI error - An IPMI error has occurred that FreeIPMI does not know how to handle. Please e-mail freeipmi-users@gnu.org to report the issue. ___ Freeipmi-users mailing list Freeipmi-users@gnu.org https://lists.gnu.org/mailman/listinfo/freeipmi-users -- Albert Chu ch...@llnl.gov Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory ___ Freeipmi-users mailing list Freeipmi-users@gnu.org https://lists.gnu.org/mailman/listinfo/freeipmi-users