Hi, Dmitry. What version of Riak are you using? And is there anything interesting in the error logs?
-Justin On Thu, Oct 14, 2010 at 7:53 AM, Dmitry Demeshchuk <[email protected]> wrote: > A small update. I've just encountered the same problem. Just about 3-4 > hours have passed. > > lsof | wc -l showed only about 2k descriptors for all users. That's > even more weird as the 32k descriptors limit is per user. So, we > haven't reached the limit so far. > > On Thu, Oct 14, 2010 at 3:48 PM, Dmitry Demeshchuk <[email protected]> > wrote: >> Greetings. >> >> We have recently started to get the emfile errors. ulimit -n is 32767. >> Restarting Riak helps for several hours and then we run out of >> descriptors again. >> >> Some time later after restart I performed lsof and found the following >> descriptors: >> >> kondemand 154 root cwd unknown >> /proc/154/cwd (readlink: Permission denied) >> kondemand 154 root rtd unknown >> /proc/154/root (readlink: Permission denied) >> kondemand 154 root txt unknown >> /proc/154/exe (readlink: Permission denied) >> kondemand 154 root NOFD >> /proc/154/fd (opendir: Permission denied) >> kondemand 155 root cwd unknown >> /proc/155/cwd (readlink: Permission denied) >> kondemand 154 root cwd unknown >> /proc/154/cwd (readlink: Permission denied) >> kondemand 154 root rtd unknown >> /proc/154/root (readlink: Permission denied) >> kondemand 154 root txt unknown >> /proc/154/exe (readlink: Permission denied) >> kondemand 154 root NOFD >> /proc/154/fd (opendir: Permission denied) >> kondemand 155 root cwd unknown >> /proc/155/cwd (readlink: Permission denied) >> kondemand 155 root rtd unknown >> /proc/155/root (readlink: Permission denied) >> kondemand 155 root txt unknown >> /proc/155/exe (readlink: Permission denied) >> kondemand 155 root NOFD >> /proc/155/fd (opendir: Permission denied) >> kondemand 156 root cwd unknown >> /proc/156/cwd (readlink: Permission denied) >> kondemand 156 root rtd unknown >> /proc/156/root (readlink: Permission denied) >> kondemand 156 root txt unknown >> /proc/156/exe (readlink: Permission denied) >> kondemand 156 root NOFD >> /proc/156/fd (opendir: Permission denied) >> kondemand 157 root cwd unknown >> /proc/157/cwd (readlink: Permission denied) >> kondemand 157 root rtd unknown >> /proc/157/root (readlink: Permission denied) >> kondemand 157 root txt unknown >> /proc/157/exe (readlink: Permission denied) >> kondemand 157 root NOFD >> /proc/157/fd (opendir: Permission denied) >> kondemand 158 root cwd unknown >> /proc/158/cwd (readlink: Permission denied) >> kondemand 158 root rtd unknown >> /proc/158/root (readlink: Permission denied) >> kondemand 158 root txt unknown >> /proc/158/exe (readlink: Permission denied) >> >> Also, the following couple of descriptors is opened several times at >> the same time: >> >> bash 20176 dem mem REG 252,0 256316 1179925 >> /usr/lib/locale/en_US.utf8/LC_CTYPE >> bash 20176 dem mem REG 252,0 54 1179926 >> /usr/lib/locale/en_US.utf8/LC_NUMERIC >> bash 20176 dem mem REG 252,0 2454 1179927 >> /usr/lib/locale/en_US.utf8/LC_TIME >> bash 20176 dem mem REG 252,0 966938 1179928 >> /usr/lib/locale/en_US.utf8/LC_COLLATE >> bash 20176 dem mem REG 252,0 286 1179929 >> /usr/lib/locale/en_US.utf8/LC_MONETARY >> bash 20176 dem mem REG 252,0 52 1179930 >> /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES >> bash 20176 dem mem REG 252,0 34 1179931 >> /usr/lib/locale/en_US.utf8/LC_PAPER >> bash 20176 dem mem REG 252,0 77 1179932 >> /usr/lib/locale/en_US.utf8/LC_NAME >> bash 20176 dem mem REG 252,0 155 1179933 >> /usr/lib/locale/en_US.utf8/LC_ADDRESS >> bash 20176 dem mem REG 252,0 59 1179934 >> /usr/lib/locale/en_US.utf8/LC_TELEPHONE >> bash 20176 dem mem REG 252,0 23 1179935 >> /usr/lib/locale/en_US.utf8/LC_MEASUREMENT >> bash 20176 dem mem REG 252,0 26048 917676 >> /usr/lib/gconv/gconv-modules.cache >> bash 20176 dem mem REG 252,0 373 1179936 >> /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION >> >> Version of Riak is 0.12.1. There was a similar problem once and the >> user was advised to make sure to use 0.12.1 >> >> Any ideas? >> >> -- >> Best regards, >> Dmitry Demeshchuk >> > > > > -- > Best regards, > Dmitry Demeshchuk > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
