RE: [CEPH] OSD daemons running with a large number of threads

2015-11-19 Thread ghislain.chevalier
Hi,

Thanks for the advice

Brgds


-Message d'origine-
De : Sage Weil [mailto:s...@newdream.net] 
Envoyé : mardi 17 novembre 2015 14:41
À : CHEVALIER Ghislain IMT/OLPS
Cc : ceph-devel@vger.kernel.org
Objet : Re: [CEPH] OSD daemons running with a large number of threads

On Tue, 17 Nov 2015, ghislain.cheval...@orange.com wrote:
> Hi,
> 
> Context:
> Firefly 0.80.9
> Ubuntu 14.04.1
> Almost a production platform  in an openstack environment
> 176 OSD (SAS and SSD), 2 crushmap-oriented storage classes , 8 servers 
> in 2 rooms, 3 monitors on openstack controllers
> Usage: Rados Gateway for object service and RBD as back-end for Cinder 
> and Glance
> 
> Issue:
> We are currently running performances tests on this cluster before turning it 
> to production.
> We created cinder volumes (attached to Ceph Back End) on virtual machines and 
> we use FIO to stress the cluster.
> A very large number of threads are created per OSD daemon (about 1000).

This is normal.  The init scripts set the max open files ulimit to a high value 
(usually 4194304 to avoid any possibility of hitting it) but you may need to 
set /proc/sys/kernel/pid_max to something big if your cluster is large.

sage

_

Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou 
falsifie. Merci.

This message and its attachments may contain confidential or privileged 
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete 
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been 
modified, changed or falsified.
Thank you.

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [CEPH] OSD daemons running with a large number of threads

2015-11-17 Thread Sage Weil
On Tue, 17 Nov 2015, ghislain.cheval...@orange.com wrote:
> Hi,
> 
> Context:
> Firefly 0.80.9
> Ubuntu 14.04.1
> Almost a production platform  in an openstack environment
> 176 OSD (SAS and SSD), 2 crushmap-oriented storage classes , 8 servers in 2 
> rooms, 3 monitors on openstack controllers
> Usage: Rados Gateway for object service and RBD as back-end for Cinder and 
> Glance
> 
> Issue:
> We are currently running performances tests on this cluster before turning it 
> to production.
> We created cinder volumes (attached to Ceph Back End) on virtual machines and 
> we use FIO to stress the cluster.
> A very large number of threads are created per OSD daemon (about 1000).

This is normal.  The init scripts set the max open files ulimit to a high 
value (usually 4194304 to avoid any possibility of hitting it) but you may 
need to set /proc/sys/kernel/pid_max to something big if your cluster is 
large.

sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html