Hi Dan,
Great! Thanks for the feedback. Much appreciated.
For completion here is my ansible role:
- name: Increase aio-max-nr for bluestore
sysctl:
name: fs.aio-max-nr
value: 1048576
sysctl_file: /etc/sysctl.d/ceph-tuning.conf
sysctl_set: yes
Cheers,
Tom
On Wed, Aug 30, 2017 at 10:49 AM, Dan van der Ster <[email protected]>
wrote:
> Hi Thomas,
>
> Yes we set it to a million.
> From our puppet manifest:
>
> # need to increase aio-max-nr to allow many bluestore devs
> sysctl { 'fs.aio-max-nr': val => '1048576' }
>
> Cheers, Dan
>
>
> On Aug 30, 2017 9:53 AM, "Thomas Bennett" <[email protected]> wrote:
> >
> > Hi,
> >
> > I've been testing out Luminous and I've noticed that at some point the
> number of osds per nodes was limited by aio-max-nr. By default its set to
> 65536 in Ubuntu 16.04
> >
> > Has anyone else experienced this issue?
> >
> > fs.aio-nr currently sitting at 196608 with 48 osds.
> >
> > I have 48 osd's per node so I've added fs.aio-max-nr = 262144 in my
> /etc/systctl.d/ceph-tuning.conf file.
> >
> > Cheers,
> > Tom
> >
> > _______________________________________________
> > ceph-users mailing list
> > [email protected]
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
--
Thomas Bennett
SKA South Africa
Science Processing Team
Office: +27 21 5067341
Mobile: +27 79 5237105
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
