Re: [PVE-User] OOM Killer problem

[Michael Rasmussen]

Latest pve kernel has a fix for a serious oom killer bug. I would try upgrading 
your kernel before anything else.

On February 4, 2017 3:16:05 PM GMT+01:00, Michele Bonera  
wrote:
>On 04/02/2017 12:35, Alwin Antreich wrote:
>
>> Hi Michele,
>>
>> On 02/04/2017 10:44 AM, Michele Bonera wrote:
>>> Hi.
>>>
>>> I have an issue with OOM Killer (Proxmox 4.4-5 - Kernel
>4.4.35-1-pve) on
>>> my infrastructure: even if there is a lot of free memory (15GB used
>over
>>> 32GB available), OOM Killer is still killing my VM processes.
>>
>> Are you over committing your machines?
>
>Not at all: the total amount of RAM assigned to the VMs running on this
>
>node is 12GB (and the host has 32GB).
>
>> Another consideration could be NUMA, if one node runs out of memory
>the
>> kernel might also kill processes.
>
>This is plausible. I'll do some checks... Thanks for the hint.
>
>>> This morning I had 3 over 6 VM powered down because OOM Killer
>decided
>>> to kill the related processes even if there were a lot of free
>memory.
>>>
>>>   total   used   free sharedbuffers cached
>>> Mem:  32725572   32286280 439292 1964283772084  
>18648708
>>> -/+:   9865488   22860084
>>> Swap:  1499124  337361465388
>>
>> It could also be possible that your swap space can't be utilized
>> properly due to your workload.
>
>I exclude this: the cpu utilization rarely exceed 20%
>
>Bye
>-- 
>Michele Bonera
>www.zipoware.com
>___
>pve-user mailing list
>pve-user@pve.proxmox.com
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.



This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.


___
pve-user mailing list
pve-user@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] OOM Killer problem

[Michele Bonera]

On 04/02/2017 12:35, Alwin Antreich wrote:


Hi Michele,

On 02/04/2017 10:44 AM, Michele Bonera wrote:

Hi.

I have an issue with OOM Killer (Proxmox 4.4-5 - Kernel 4.4.35-1-pve) on
my infrastructure: even if there is a lot of free memory (15GB used over
32GB available), OOM Killer is still killing my VM processes.


Are you over committing your machines?


Not at all: the total amount of RAM assigned to the VMs running on this 
node is 12GB (and the host has 32GB).



Another consideration could be NUMA, if one node runs out of memory the
kernel might also kill processes.


This is plausible. I'll do some checks... Thanks for the hint.


This morning I had 3 over 6 VM powered down because OOM Killer decided
to kill the related processes even if there were a lot of free memory.

  total   used   free sharedbuffers cached
Mem:  32725572   32286280 439292 1964283772084   18648708
-/+:   9865488   22860084
Swap:  1499124  337361465388


It could also be possible that your swap space can't be utilized
properly due to your workload.


I exclude this: the cpu utilization rarely exceed 20%

Bye
--
Michele Bonera
www.zipoware.com
___
pve-user mailing list
pve-user@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] OOM Killer problem

[Falco Kleinschmidt]

I had the same problems regarding KVMs not running and I thought that
they were gone, but I had again some problems.

Yesterday I rebooted after an upgrade and now I am running VE 4.4-12 /
kernel 4.4.35-2-pve, hoping that no stopped KVMs will occure again.

I am using the following cronscript every minute to check if all KVMs
are running. If not, the script starts the KVM. Safed me some customers
calling.

#!/bin/bash
###
#If KVM machines are stopped, they will be started
###
for VM in `/usr/sbin/qm list | grep stopped | awk '{print $1}'`; do
  echo `date`" VM $VM is offline, starting..." >> /var/log/check-kvm.log
  /usr/sbin/qm start $VM
done



Am 04.02.2017 um 10:44 schrieb Michele Bonera:
> Hi.
>
> I have an issue with OOM Killer (Proxmox 4.4-5 - Kernel 4.4.35-1-pve)
> on my infrastructure: even if there is a lot of free memory (15GB used
> over 32GB available), OOM Killer is still killing my VM processes.
>
> This morning I had 3 over 6 VM powered down because OOM Killer decided
> to kill the related processes even if there were a lot of free memory.
>
>   total   used   free sharedbuffers cached
> Mem:  32725572   32286280 439292 1964283772084   18648708
> -/+:   9865488   22860084
> Swap:  1499124  337361465388
>
> This problem is related to all the server I have in the infrastructure
> (some are Supermicro, some are Dell) and occur randomly after some
> days the host is rebooted.
>
> I already instruct OOM Killer not to kill VM Processes, but I'm afraid
> it could kill processes related to the infrastructure.
>
> Is there someone with the same issue?
>
> Bye,

-- 


Datenfalke - Dipl. Inf. Falco Kleinschmidt
Adresse: Dinnendahlstr. 8 - 45136 Essen
Steuer-Nr: DE248267798
Telefon: +49-(0)201-6124650
Fax: +49-(0)201-6124651
Email: f...@datenfalke.de
WWW: http://www.datenfalke.de

___
pve-user mailing list
pve-user@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] OOM Killer problem

[Alwin Antreich]

Hi Michele,

On 02/04/2017 10:44 AM, Michele Bonera wrote:
> Hi.
> 
> I have an issue with OOM Killer (Proxmox 4.4-5 - Kernel 4.4.35-1-pve) on
> my infrastructure: even if there is a lot of free memory (15GB used over
> 32GB available), OOM Killer is still killing my VM processes.

Are you over committing your machines?

Another consideration could be NUMA, if one node runs out of memory the
kernel might also kill processes.

> 
> This morning I had 3 over 6 VM powered down because OOM Killer decided
> to kill the related processes even if there were a lot of free memory.
> 
>   total   used   free sharedbuffers cached
> Mem:  32725572   32286280 439292 1964283772084   18648708
> -/+:   9865488   22860084
> Swap:  1499124  337361465388

It could also be possible that your swap space can't be utilized
properly due to your workload.

> 
> This problem is related to all the server I have in the infrastructure
> (some are Supermicro, some are Dell) and occur randomly after some days
> the host is rebooted.
> 
> I already instruct OOM Killer not to kill VM Processes, but I'm afraid
> it could kill processes related to the infrastructure.
> 
> Is there someone with the same issue?
> 
> Bye,

--
Cheers,
Alwin
___
pve-user mailing list
pve-user@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] looking for recommendations of VLAN setup

[Alwin Antreich]

Hi Uwe,

On 02/02/2017 10:22 AM, Uwe Sauter wrote:
> Hi all,
> 
> I would like to hear recommendations regarding the network setup of a Proxmox 
> cluster. The situation is the following:
> 
> * Proxmox hosts have several ethernet links
> * multiple VLANs are used in our datacenter
> * I cannot guarantee that the VLANs are on the same interface for each host 
> (meaning VLAN 1234 could be on eth1 for host A but on
> eth3 for host B, e.g. due to performance reasons)
> * passing through a VLAN trunk to a VM currently doesn't seem to be necessary
> 
> My thought was that I would need to create a VLAN interface for each used 
> VLAN (say ethX.1234) and on top of that configure a
> Linux bridge (vmbr1234). This should abstract away the physical network 
> situation from the VMs.
> 
> What kind of network setup would you recommend?

We also use multiple VLANs on our network. As linux bridges are
VLAN-aware (bridge-vlan-aware yes), we set the VLAN in the VM config and
leave the interface without any VLAN (ethX), all traffic of the VM is
passing as tagged through the bridge. You only need to make sure that
you have the same bridge configured (vmbrX -> bridge-vlan-aware yes) on
all Proxmox hosts.

Eg:

Node A (eth0 -> vmbr0) - switch port VLAN 4,7,12
Node B (eth3 -> vmbr0) - switch port VLAN 3,4,7
Node C (bond0 -> vmbr0)- switch port VLAN 4,5,6

So all VMs can move between nodes, as the bridge is the same. Traffic
between VMs and devices outside the node will only happen if VLANs are
corresponding.

> 
> One additional question: does Proxmox stop me from migration a VM to a host 
> that doesn't provide VLANs necessary to that
> particular VM? (e.g. a VM needs VLAN 1234 but some of the hosts aren't 
> connected to this VLAN)
> 
> 
> Regards,
> 
>   Uwe
> ___
> pve-user mailing list
> pve-user@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 
--
Cheers,
Alwin
___
pve-user mailing list
pve-user@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[PVE-User] OOM Killer problem

[Michele Bonera]

Hi.

I have an issue with OOM Killer (Proxmox 4.4-5 - Kernel 4.4.35-1-pve) on 
my infrastructure: even if there is a lot of free memory (15GB used over 
32GB available), OOM Killer is still killing my VM processes.


This morning I had 3 over 6 VM powered down because OOM Killer decided 
to kill the related processes even if there were a lot of free memory.


  total   used   free sharedbuffers cached
Mem:  32725572   32286280 439292 1964283772084   18648708
-/+:   9865488   22860084
Swap:  1499124  337361465388

This problem is related to all the server I have in the infrastructure 
(some are Supermicro, some are Dell) and occur randomly after some days 
the host is rebooted.


I already instruct OOM Killer not to kill VM Processes, but I'm afraid 
it could kill processes related to the infrastructure.


Is there someone with the same issue?

Bye,
--
Michele Bonera
www.zipoware.com
___
pve-user mailing list
pve-user@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user