Hmm, log handling is a bit outside my expertise. You should be able to get a "config dump" from the OSD's admin socket; can you paste that? (It includes the log file locations and log levels.) If you restart a particular OSD, does it write anything to its log file at that time? -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com
On Mon, Apr 28, 2014 at 10:14 PM, Indra Pramana <[email protected]> wrote: > Hi Greg, > > Yes, all my OSDs are running. > > 1945 ? Ssl 215:36 /usr/bin/ceph-osd --cluster=ceph -i 12 -f > 2090 ? Sl 165:07 /usr/bin/ceph-osd --cluster=ceph -i 15 -f > 2100 ? Sl 205:29 /usr/bin/ceph-osd --cluster=ceph -i 13 -f > 2102 ? Sl 196:01 /usr/bin/ceph-osd --cluster=ceph -i 14 -f > > I didn't change log rotation settings from the default. This happens to all > my OSD nodes, not only this one. > > Is there a way I can verify if the logs are actually being written by the > ceph-osd processes? > > Looking forward to your reply, thank you. > > Cheers. > > > > On Tue, Apr 29, 2014 at 12:28 PM, Gregory Farnum <[email protected]> wrote: >> >> Are your OSDs actually running? I see that your older logs have more data >> in them; did you change log rotation from the defaults? >> >> >> On Monday, April 28, 2014, Indra Pramana <[email protected]> wrote: >>> >>> Hi Greg, >>> >>> The log rotation works fine, it will rotate the logs every day at around >>> 6:50am. However, there are no writes to the files (except for one osd log >>> file) so it will rotate empty files for most of them. >>> >>> -rw-r--r-- 1 root root 313884 Apr 29 12:07 ceph-osd.12.log >>> -rw-r--r-- 1 root root 198319 Apr 29 06:36 ceph-osd.12.log.1.gz >>> -rw-r--r-- 1 root root 181675 Apr 28 06:50 ceph-osd.12.log.2.gz >>> -rw-r--r-- 1 root root 44012 Apr 27 06:53 ceph-osd.12.log.3.gz >>> -rw-r--r-- 1 root root 0 Apr 29 06:36 ceph-osd.13.log >>> -rw-r--r-- 1 root root 20 Apr 28 06:50 ceph-osd.13.log.1.gz >>> -rw-r--r-- 1 root root 506 Apr 27 06:53 ceph-osd.13.log.2.gz >>> -rw-r--r-- 1 root root 44605 Apr 27 06:53 ceph-osd.13.log.3.gz >>> -rw-r--r-- 1 root root 0 Apr 29 06:36 ceph-osd.14.log >>> -rw-r--r-- 1 root root 20 Apr 28 06:50 ceph-osd.14.log.1.gz >>> -rw-r--r-- 1 root root 502 Apr 27 06:53 ceph-osd.14.log.2.gz >>> -rw-r--r-- 1 root root 55570 Apr 27 06:53 ceph-osd.14.log.3.gz >>> -rw-r--r-- 1 root root 0 Apr 29 06:36 ceph-osd.15.log >>> -rw-r--r-- 1 root root 20 Apr 28 06:50 ceph-osd.15.log.1.gz >>> -rw-r--r-- 1 root root 500 Apr 27 06:53 ceph-osd.15.log.2.gz >>> -rw-r--r-- 1 root root 49090 Apr 27 06:53 ceph-osd.15.log.3.gz >>> >>> Any advice? >>> >>> Thank you. >>> >>> >>> On Mon, Apr 28, 2014 at 11:26 PM, Gregory Farnum <[email protected]> >>> wrote: >>>> >>>> It is not. My guess from looking at the time stamps is that maybe you >>>> have a log rotation system set up that isn't working properly? >>>> -Greg >>>> >>>> >>>> On Sunday, April 27, 2014, Indra Pramana <[email protected]> wrote: >>>>> >>>>> Dear all, >>>>> >>>>> I have multiple OSDs per node (normally 4) and I realised that for all >>>>> the nodes that I have, only one OSD will contain logs under /var/log/ceph, >>>>> the rest of the logs are empty. >>>>> >>>>> root@ceph-osd-07:/var/log/ceph# ls -la *.log >>>>> -rw-r--r-- 1 root root 0 Apr 28 06:50 ceph-client.admin.log >>>>> -rw-r--r-- 1 root root 0 Apr 28 06:50 ceph-osd.0.log >>>>> -rw-r--r-- 1 root root 386857 Apr 28 14:02 ceph-osd.12.log >>>>> -rw-r--r-- 1 root root 0 Apr 28 06:50 ceph-osd.13.log >>>>> -rw-r--r-- 1 root root 0 Apr 28 06:50 ceph-osd.14.log >>>>> -rw-r--r-- 1 root root 0 Apr 28 06:50 ceph-osd.15.log >>>>> -rw-r--r-- 1 root root 0 Apr 28 06:50 ceph-osd..log >>>>> >>>>> The ceph-osd.12.log only contains the logs for osd.12 only, while the >>>>> other logs for osd.13, 14 and 15 are not available and empty. >>>>> >>>>> Is this normal? >>>>> >>>>> Looking forward to your reply, thank you. >>>>> >>>>> Cheers. >>>> >>>> >>>> >>>> -- >>>> Software Engineer #42 @ http://inktank.com | http://ceph.com >>> >>> >> >> >> -- >> Software Engineer #42 @ http://inktank.com | http://ceph.com > > _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
