Re: Odd values for various memory metrics via SNMP
Hi Yuri, first thanks a lot for your help and insight. Additionally I had a short exchange with Michael W. Lucas and I am also studying his SNMP Mastery book at the moment. > OK, guess the following (physical memory) looks correct for you as well? Yes, physical is correct across all hosts - FreeBSD, Linux, ESXi, ... > So I guess what net-snmp reports makes sense and the only value that > should be used for alerts is that "physical memory" one. I learned from MWL that "real memory" is supposed to be all memory currently in use by applications. Allocated pages. That of course makes sense, too. Now only "virtual memory" is left - according to the book that should be the sum of physical memory and swap space - and for my single Linux host it is. For the FreeBSD systems this is the one left that still looks nonsensical. OPNsense (8G RAM, 8G swap): 4.91 of 4.99G used. TrueNAS (64G RAM, 32G swap): 627 of 628G used. TrueNAS 2 (32G RAM, 32G swap): 512 of 516G used. Recompiling net-snmp is not an option for me for these appliance OSes, but knowing that at least the "real memory" values do make sense, I'll try to teach Observium not to paint them bright red. Kind regards, Patrick -- punkt.de GmbH Patrick M. Hausen infrastructure Sophienstr. 187 76185 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de i...@punkt.de AG Mannheim 108285 Geschäftsführer: Daniel Lienert, Fabian Stein
Re: Odd values for various memory metrics via SNMP
HI Yuri, > Am 19.12.2023 um 09:50 schrieb Yuri : > Let's start with OIDs, which ones exactly you are looking at (numeric or > textual will do)? To start I'll post the 4 odd ones of my OPNsense system, FreeBSD 13.2-p7, net-snmp 5.9.1_4,1: HOST-RESOURCES-MIB::hrStorageIndex.2 = INTEGER: 2 HOST-RESOURCES-MIB::hrStorageType.2 = OID: HOST-RESOURCES-TYPES::hrStorageRam HOST-RESOURCES-MIB::hrStorageDescr.2 = STRING: Real memory HOST-RESOURCES-MIB::hrStorageAllocationUnits.2 = INTEGER: 4096 Bytes HOST-RESOURCES-MIB::hrStorageSize.2 = INTEGER: 191414 HOST-RESOURCES-MIB::hrStorageUsed.2 = INTEGER: 189988 HOST-RESOURCES-MIB::hrStorageAllocationFailures.2 = Counter32: 0 --- HOST-RESOURCES-MIB::hrStorageIndex.9 = INTEGER: 9 HOST-RESOURCES-MIB::hrStorageType.9 = OID: HOST-RESOURCES-TYPES::hrStorageOther HOST-RESOURCES-MIB::hrStorageDescr.9 = STRING: Shared real memory HOST-RESOURCES-MIB::hrStorageAllocationUnits.9 = INTEGER: 4096 Bytes HOST-RESOURCES-MIB::hrStorageSize.9 = INTEGER: 25627 HOST-RESOURCES-MIB::hrStorageUsed.9 = INTEGER: 24175 HOST-RESOURCES-MIB::hrStorageAllocationFailures.9 = Counter32: 0 --- HOST-RESOURCES-MIB::hrStorageIndex.8 = INTEGER: 8 HOST-RESOURCES-MIB::hrStorageType.8 = OID: HOST-RESOURCES-TYPES::hrStorageOther HOST-RESOURCES-MIB::hrStorageDescr.8 = STRING: Shared virtual memory HOST-RESOURCES-MIB::hrStorageAllocationUnits.8 = INTEGER: 4096 Bytes HOST-RESOURCES-MIB::hrStorageSize.8 = INTEGER: 54962 HOST-RESOURCES-MIB::hrStorageUsed.8 = INTEGER: 41268 HOST-RESOURCES-MIB::hrStorageAllocationFailures.8 = Counter32: 0 --- HOST-RESOURCES-MIB::hrStorageIndex.3 = INTEGER: 3 HOST-RESOURCES-MIB::hrStorageType.3 = OID: HOST-RESOURCES-TYPES::hrStorageVirtualMemory HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: Virtual memory HOST-RESOURCES-MIB::hrStorageAllocationUnits.3 = INTEGER: 4096 Bytes HOST-RESOURCES-MIB::hrStorageSize.3 = INTEGER: 1279129 HOST-RESOURCES-MIB::hrStorageUsed.3 = INTEGER: 1258824 HOST-RESOURCES-MIB::hrStorageAllocationFailures.3 = Counter32: 0 Kind regards, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Sophienstr. 187 76185 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de i...@punkt.de AG Mannheim 108285 Geschäftsführer: Daniel Lienert, Fabian Stein
Odd values for various memory metrics via SNMP
Hi all, what's the best platform/list/forum/... to discuss issues with the SNMP implementations for FreeBSD? Since I started to dig deeper into Obervium I found that of all the systems I own only the FreeBSD based ones present some challenges regarding the interpretation of memory metrics. 1. I confirmed that this is not an Observium artefact. LibreNMS and manual snmpwalk give the same results. 2. This is independent of the question if you use net-snmp (FreeNAS, OPNsense) or bsnmpd in base (my Raspberry Pi cluster). 3. These are the systems concerned: - OPNsense - FreeBSD 13.2, AMD64, 8 G of RAM, net-snmp - FreeNAS - TrueANS CORE, FreeBSD 13.1, AMD64, 64 G of RAM, net-snmp - 7 Raspberry Pi CM 3+, AARCH64, 1 G of RAM, bsnmpd 4. The "odd" metrics: - "Real Memory" OPNsense: 762 of 768 M (yes!) used, flagged "red" in Observium, of course FreeNAS: 35.9 of 36.1 G used All the Pis: 178 of 179 M used All - see hardware description above - no connection to the "real real" memory installed. For comparison: ESXi 8.0: 26.7 of 31.9 G used - the system has got 32 G of RAM installed. - "Shared real memory" OPNsense: 97.1 of 103 M used FreeNAS: 593 of 774 M used Pis: 11.8 of 12.4 M used - Shared virtual memory Only systems running net-snmp have this, bsnmpd based ones don't. OPNsense: 166 of 233 M used FreeNAS: 1028.27 M of 1.49 G used - Virtual Memory Again, only via net-snmp. OPNsense: 5.03 of 5.11 G used FreeNAS: 628 of 628 G used - that's particularly weird for a system with 64 G of RAM installed and not swapping or anything. For comparison: my only Linux based system: 12.7 of 29.6 G used - the system has 16 G of RAM plus 16 G of swap, that seems to match the "29.6". For me these numbers don't make any sense at all. The motivation to write this email: I am planning to use SNMP based monitoring, probably Observium, for all our data centres, which means > 100 FreeBSD based hosts. The point of a monitoring/management system is that anything flagged "red" is a real problem that needs attention and the default state should be "everything super green". Things flagged "red" but "everybody knows it can be ignored" bind a huge amount of brain capacity on behalf of the operators. Not good. So what is going on here? What do these numbers actually mean? Where do they come from? Are they artefacts of the SNMP implementation not taylored perfectly for FreeBSD or are they some real metric that ends up interpreted wrong in the NMS (Observium)? Thanks and kind regards, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Sophienstr. 187 76185 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de i...@punkt.de AG Mannheim 108285 Geschäftsführer: Daniel Lienert, Fabian Stein
