Going back to the OP, here's a postmortem on exactly what happened:
In short, we had unknowingly downgraded from 64 bit to 32 bit during an
upgrade. It's covered here:
https://doc.pfsense.org/index.php/Upgrade_Guide#Avoiding_Unintended_Architecture_Change
Interestingly, it worked like this:
We were running 64 bit 2.2.6. Working fine. We installed 64-bit 2.3.2,
and restored our config. Again, working fine. Then we upgraded to
*in-place* to 2.3.2_1, which for the *first time*, referenced a vestigal
"upgrade URL" in our config, which had been carried-forward from an era
when this was 'normal'. Because the old URL happened to point to a 32
bit architecture (the only architecture back then), we were unknowingly
downgraded to the 32 bit architecture without realizing it. And of
course, the 32 bit defaults are VERY different from the 64 bit version
(IIRC ~5x greater kern.ipc.nmbclusters and kern.ipc.nmbufs for 64 bit)
causing our observed kernel panic on the ample Supermicro A1SRi-2758F
(with its huge pile of cores and IGB NIC's) etc.. The pfSense project
has correctly set stingy defaults in the 32 bit architecture, and
there's no case to be made for running the 32 bit stack on a beefy board
like this one.
I suspect many/most of people who have written in forum posts about how
pfsense doesn't 'Just work' on the popular Supermicro A1SRi-2758F are
running the 32 bit stack or unknowingly downgrading.
Last night we manually edited the config, and restored it to a clean
64-bit image of 2.3.2, and as expected, it 'just worked' with no sysctrl
modifications. The upgrade to 2.3.2_1 was also flawless because the old
upgrade URL had been removed from the config.
On 1/25/2017 4:01 PM, Karl Fife wrote:
This is a good theory, because RRD data from 2.2.6 suggests that the
difference in utilization between the versions is slight, and that we
had 'barely' exhausted our system default allocation.
Is there a difference between nano and full with respect to the
installer explicitly setting tunables for kern.ipc.nmbclusters and
kern.ipc.nmbuf? Vick Khera says he sees explicitly set tunables on
his 2.3.2 system, yet my virgin installation of Nano pfSense 2.3.2 has
no explicit declarations?
Vick, is your Supermicro A1SRi-2758F running an installation that came
from Netgate, or is it a community edition installation? If the
latter, Full or Nano?
On 1/25/2017 3:49 PM, Jim Pingle wrote:
On 01/25/2017 01:10 PM, Karl Fife wrote:
The piece that's still missing for me is that there must have been some
change in default system setting for FreeBSD, or some other change
between versions, because the system booted fine with pfSense v 2.2.6
Aside from what has already been suggested by others, it's possible that
the newer drivers from FreeBSD 10.3 in pfSense 2.3.x enabled features on
the NIC chipset that consumed more mbufs. For example, it might be using
more queues per NIC by default than it did previously.
Jim
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
