We're on vSphere 6. But it seems unlikely that the vmxnet3 adapter is at the root of this, as the hosts and VMs are well-established (almost 3 years), and the upgrade from 5.5 took place over a year ago.
The only major change adjacent to this problem involved moving to the Nimble, and migrating all of the VMs away from the EMC VNX5400 and VNXe3100. They (all SANS and all hosts) are connected to a stacked pair of Juniper EX 4300s, but we did add in 4-port 10g SFP moduled and cabled the Nimble to that. Even then, it wasn't until we were a couple of weeks into the migration that we started seeing this problem. I'm willing to believe that it's the Junipers, but I want to get VMware sussed out before I head there. I say that because I haven't yet deleted the VMNICs for the EMCs - we kept the same VLAN, but migrated the address space in the VLAN (it's isolated) from 10.10.0.0/14 to 10.211.10.0/24, as the 10.10.0.0/24 space took a chunk out of our lab's address space. Kurt On Wed, May 31, 2017 at 4:55 PM, Don Ely <[email protected]> wrote: > What version of vSphere? There are some known issues with the vmxnet3 > adapter > > On May 31, 2017 4:51 PM, "Kurt Buff" <[email protected]> wrote: >> >> Update - still not solved: >> >> Got on a call with a MSFT rep. He ran a quick shell script that did >> the things I've already done: >> >> netsh interface tcp set global chimney=disabled >> netsh interface tcp set global rss=disabled >> netsh int tcp set global autotuning=disabled >> netsh int tcp set global congestion=none >> netsh int tcp set global netdma=Disabled >> >> I've put him off for now, as I'm seeing what might be a related >> problem crop up - the ancient CRM we're using has been spouting errors >> all day about not being able to write to the database. >> >> I've looked at CPU ready on both machines, and the file server's is >> pretty bad, but the other server's isn't. That's after migrating them >> to a single host together, and migrating everything else off that host >> - just those two VMs on this host. I've also looked at performance >> charts in vmware for both machines regarding disk and network, and am >> not seeing anything out of line. >> >> I'm trying to install the vmware support assistant appliance, but am >> running into problems with SSO auth - the vsphere infrastructure was >> upgraded from 5.5 to 6.0, and it looks like I have a project ahead of >> me to fix the SSL certs, which this post seems to cover: >> >> https://virtuallyunderstood.wordpress.com/2016/08/03/troubleshooting-expired-psc-certificates-with-vsphere-6/ >> >> Further, I've checked with Nimble support, and they say that there is >> some latency, but that their tools indicate that it is external to the >> array - they're pointing at vsphere or the network, and suggesting I >> should fail over the array to its other interface to see if that >> clears the problem. I'm saving that for later. >> >> I'm also going to see about setting up a machine to monitor the >> server/iSCSI switch to which the hosts and SANs are attached - what >> I'm seeing in PRTG for that doesn't give me what I want. >> >> It just goes deeper and deeper... >> >> Kurt >> >> Kurt >> >> >> I've got a ticket open with vmware now >> >> On Fri, May 26, 2017 at 12:20 PM, Kurt Buff <[email protected]> wrote: >> > All, >> > >> > I have a 2012R2 file server running as a VM on vSphere 6.0. >> > >> > Here's what I'm seeing: >> > >> > Copy large file (win7 ISO) from file server to workstation, I get >> > roughly 12-13Mbytes/second, wired or wireless. >> > >> > Copy that file from workstation to server over a wireless connection, >> > same speed - 12-13Mbytes/second >> > >> > Copy that file from workstation to server over wired connection, speed >> > degrades to 1Mbyte/second or less >> > >> > Copy that file to another 2012R2 VM on the same host on the same SAN >> > volume (our print server), and speeds are 12-13Mbytes/second for both >> > wired and wireless. >> > >> > I've made sure that the following are disabled: RSS, atime, 8.3 >> > filename generation, TCP Chimney. >> > >> > RAM and CPU utilization on this machine are well within limits. >> > >> > I'm thoroughly stumped. >> > >> > Anyone have pointers for me? I'm about to raise a case with MSFT. >> > >> > Kurt >> >> >
