Re: [gentoo-user] Debug memory leaks in X server
On Monday 26 Nov 2012 09:56:27 Florian Philipp wrote: Hi list! I have a suspicion that viewing certain PDFs in okular causes X server to leak memory. Currently it is using 1.8 GB after 3 days uptime. Has anyone else observed that? Is there a way to inspect X server's memory usage? I have noticed that okular recently started breaking into a sweat when it renders pdf files. Even a four page document with a bit of colour and graphics seems to push the cpu: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 16387 michael 23 3 466m 52m 30m S 102 1.3 0:36.88 /usr/bin/okular In case someone can interpret the content of the file: = $ mupdfinfo Downloads/E016378510.pdf Downloads/E016378510.pdf: PDF-1.4 Info object (18 0 R): /CreationDate (D:20121126030121-05'00') /Subject /Author /Creator (PlanetPress Watch) /Keywords /Producer (Distiller) /ModDate (D:20121126030123-05'00') /Title (CONFIRMATIONS) Pages: 4 Retrieving info from pages 1-4... Mediaboxes (1): 1 ( 21 0 R): [ 0 0 612 792 ] Fonts (2): 1 ( 21 0 R): TrueType 'FIBAKI+GillSans' (23 0 R) 1 ( 21 0 R): Type1 'GillSans-Bold' (28 0 R) Images (7): 1 ( 21 0 R): [ DCT ] 1125x1050 8bpc DevGray (31 0 R) 1 ( 21 0 R): [ DCT ] 2550x3300 8bpc ICC (32 0 R) 1 ( 21 0 R): [ Flate ] 481x164 8bpc ICC (33 0 R) 2 ( 1 0 R): [ DCT ] 2550x3300 8bpc ICC (4 0 R) 2 ( 1 0 R): [ DCT ] 1125x1050 8bpc DevGray (5 0 R) 3 ( 6 0 R): [ DCT ] 2550x3300 8bpc ICC (9 0 R) 4 ( 10 0 R): [ DCT ] 2549x3300 8bpc ICC (13 0 R) = Once rendered, the CPU goes down to normal levels. The problem does not seem to occur when the pdf is just text, i.e. no photographs, or complex graphics in it. Other than the various top apps, perhaps you can try lsof? -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Debug memory leaks in X server
Mick wrote: On Monday 26 Nov 2012 09:56:27 Florian Philipp wrote: Hi list! I have a suspicion that viewing certain PDFs in okular causes X server to leak memory. Currently it is using 1.8 GB after 3 days uptime. Has anyone else observed that? Is there a way to inspect X server's memory usage? I have noticed that okular recently started breaking into a sweat when it renders pdf files. Even a four page document with a bit of colour and graphics seems to push the cpu: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 16387 michael 23 3 466m 52m 30m S 102 1.3 0:36.88 /usr/bin/okular SNIP Once rendered, the CPU goes down to normal levels. The problem does not seem to occur when the pdf is just text, i.e. no photographs, or complex graphics in it. Other than the various top apps, perhaps you can try lsof? I have a local grocery store that I have to go to the website to get their sale ads. Anyway, it is generally 2 pages and even on this 4 core rig with more than plenty of ram, it takes a bit to open them. Funny thing is, I have some that is about sewing, lots of pictures in those since I need pics to get the idea, anyway, they load up in a flash. As soon as Okular loads, the pages are there. Since this is more like what Florian describes, I guess we see the same things. I'm not sure about ram itself but some files do open differently. By the way, the grocery ad is a much smaller file than the sewing files. Both in file size and number of pages. One would expect it to be the opposite. Looks like I have a problem that I didn't know I had. With 16Gbs of ram, I hadn't noticed anything with the ram, other than Seamonkey being its usual hoggy self. :/ I guess this is to sort of confirm that someone else sees a similar thing to Florian. Dale :-) :-) -- I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
Re: [gentoo-user] Debug memory leaks in X server
On Monday 26 Nov 2012 12:43:38 Dale wrote: Mick wrote: On Monday 26 Nov 2012 09:56:27 Florian Philipp wrote: Hi list! I have a suspicion that viewing certain PDFs in okular causes X server to leak memory. Currently it is using 1.8 GB after 3 days uptime. Has anyone else observed that? Is there a way to inspect X server's memory usage? I have noticed that okular recently started breaking into a sweat when it renders pdf files. Even a four page document with a bit of colour and graphics seems to push the cpu: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 16387 michael 23 3 466m 52m 30m S 102 1.3 0:36.88 /usr/bin/okular SNIP Once rendered, the CPU goes down to normal levels. The problem does not seem to occur when the pdf is just text, i.e. no photographs, or complex graphics in it. Other than the various top apps, perhaps you can try lsof? I have a local grocery store that I have to go to the website to get their sale ads. Anyway, it is generally 2 pages and even on this 4 core rig with more than plenty of ram, it takes a bit to open them. Funny thing is, I have some that is about sewing, lots of pictures in those since I need pics to get the idea, anyway, they load up in a flash. As soon as Okular loads, the pages are there. Since this is more like what Florian describes, I guess we see the same things. I'm not sure about ram itself but some files do open differently. By the way, the grocery ad is a much smaller file than the sewing files. Both in file size and number of pages. One would expect it to be the opposite. Looks like I have a problem that I didn't know I had. With 16Gbs of ram, I hadn't noticed anything with the ram, other than Seamonkey being its usual hoggy self. :/ I guess this is to sort of confirm that someone else sees a similar thing to Florian. This is not a RAM issue, but seemingly a CPU issue. Furthermore, it does not seem to be related exclusively to okular. I just tried qpdfview and it also took ages to open/render - HOWEVER - when I tried mupdf it was rendered in milliseconds and the CPU usage stayed very very low. This may be something to do with the wonderful KDE and friends. I recently upgraded to KDE-4.9.3 and this is not the only thing I noticed. In Kmail-1.13.7 all sent messages are saved in the local/sent-mail directory, irrespective of the path I enter in Kmail's settings for each email account. Initially I though that sent messages were being lost - not sent - but then I noticed that the default sent folder started getting larger. I better start another thread for this problem. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Debug memory leaks in X server
Mick wrote: On Monday 26 Nov 2012 12:43:38 Dale wrote: Mick wrote: On Monday 26 Nov 2012 09:56:27 Florian Philipp wrote: Hi list! I have a suspicion that viewing certain PDFs in okular causes X server to leak memory. Currently it is using 1.8 GB after 3 days uptime. Has anyone else observed that? Is there a way to inspect X server's memory usage? I have noticed that okular recently started breaking into a sweat when it renders pdf files. Even a four page document with a bit of colour and graphics seems to push the cpu: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 16387 michael 23 3 466m 52m 30m S 102 1.3 0:36.88 /usr/bin/okular SNIP Once rendered, the CPU goes down to normal levels. The problem does not seem to occur when the pdf is just text, i.e. no photographs, or complex graphics in it. Other than the various top apps, perhaps you can try lsof? I have a local grocery store that I have to go to the website to get their sale ads. Anyway, it is generally 2 pages and even on this 4 core rig with more than plenty of ram, it takes a bit to open them. Funny thing is, I have some that is about sewing, lots of pictures in those since I need pics to get the idea, anyway, they load up in a flash. As soon as Okular loads, the pages are there. Since this is more like what Florian describes, I guess we see the same things. I'm not sure about ram itself but some files do open differently. By the way, the grocery ad is a much smaller file than the sewing files. Both in file size and number of pages. One would expect it to be the opposite. Looks like I have a problem that I didn't know I had. With 16Gbs of ram, I hadn't noticed anything with the ram, other than Seamonkey being its usual hoggy self. :/ I guess this is to sort of confirm that someone else sees a similar thing to Florian. This is not a RAM issue, but seemingly a CPU issue. Furthermore, it does not seem to be related exclusively to okular. I just tried qpdfview and it also took ages to open/render - HOWEVER - when I tried mupdf it was rendered in milliseconds and the CPU usage stayed very very low. This may be something to do with the wonderful KDE and friends. I recently upgraded to KDE-4.9.3 and this is not the only thing I noticed. In Kmail-1.13.7 all sent messages are saved in the local/sent-mail directory, irrespective of the path I enter in Kmail's settings for each email account. Initially I though that sent messages were being lost - not sent - but then I noticed that the default sent folder started getting larger. I better start another thread for this problem. That was mostly my point but I seemed to have glossed over the CPU part after rereading my post. My point was, ram is not a issue because I have lots of it. The subject mentions memory so I mostly went to it instead of the CPU. Even with that much ram, it takes a long time to load, at least for some files. It seems to be a CPU thing, not a memory thing. Maybe Florian knows something we don't know at this point. I don't know the exact percentages but I usually max out at least one core when I open the grocery ad. The sewing files, although larger, open faster. I just wonder what is used to make one as opposed to the other? Maybe that is the difference. Dale :-) :-) -- I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
Re: [gentoo-user] Debug memory leaks in X server
Am Montag, 26. November 2012, 10:22:17 schrieb Mick: On Monday 26 Nov 2012 09:56:27 Florian Philipp wrote: Hi list! I have a suspicion that viewing certain PDFs in okular causes X server to leak memory. Currently it is using 1.8 GB after 3 days uptime. Has anyone else observed that? Is there a way to inspect X server's memory usage? I have noticed that okular recently started breaking into a sweat when it renders pdf files. Even a four page document with a bit of colour and graphics seems to push the cpu: not here? hm.. maybe a poppler issue? -- #163933
Re: [gentoo-user] Debug memory leaks in X server
Am 26.11.2012 10:56, schrieb Florian Philipp: Hi list! I have a suspicion that viewing certain PDFs in okular causes X server to leak memory. Currently it is using 1.8 GB after 3 days uptime. Has anyone else observed that? Is there a way to inspect X server's memory usage? Regards, Florian Philipp Okay, I have now a better understanding of my memory issue (no clue about Mick's CPU issue. Maybe you can post a link to a PDF to demonstrate?): As it seems, Okular is abusing the X server to share pixmaps between instances [1]. In theory, this reduces memory consumption and CPU time, however, there are two issues: a) Sometimes Okular is leaking allocations. b) X cannot handle this usage pattern, sometimes leaks memory or cannot deallocate it because of memory fragmentation [2]. Neither side seems to be willing to fix the issue. For now, my workarounds will be: - Playing around with Okular's memory settings to make the issue less critical. - Using frontswap and normal swap + high swappiness settings to swap out the leaked memory. - Looking for another PDF/djvu/ps viewer. Maybe Wayland can handle the situation better in a few years. [1] https://bugs.kde.org/show_bug.cgi?id=177213 [2] https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/98783 BTW: Regarding my question for an X memory inspector: x11-misc/xrestop is the answer. Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
