J. Roeleveld writes:
> On Wednesday 25 August 2010 03:32:40 Alex Schuster wrote:
> > I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board
> > Radeon HD 3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM
> > volumes, all encrypted, except for /usr/src and portage stuff. The
> > system is ~amd64, and I have -march=k8-sse3 in my CFLAGS. Current
> > kernel is 2.6.34-tuxonice, but I also tried others. I'm running KDE4
> > with desktop effects enabled, X itself takes about 30-40% of CPU
> > time according to top. After system startup and login into KDE, 3.5G
> > of RAM are occupied. This increases after a while, and I need swap
> > space. Nothing to worry about I think.
>
> Encrypted filesystems can cause additional with activity, but I would
> expect that to remain the same over a long period.
And I just moved my PORTAGE_TMPDIR to an unencrypted partition.
Can LVM create noticeable overhead? I also resized my logical volumes a
couple of times, could this lead to some LVM fragmentation?
> However, how is the write and read performance on those disks?
Here's the output of hdparm -t for all drives, 4 times.
/dev/sda: (SATA system drive)
Timing buffered disk reads: 118 MB in 3.08 seconds = 38.37 MB/sec
Timing buffered disk reads: 194 MB in 3.11 seconds = 62.47 MB/sec
Timing buffered disk reads: 322 MB in 3.01 seconds = 106.82 MB/sec
Timing buffered disk reads: 244 MB in 3.00 seconds = 81.21 MB/sec
/dev/sdb: (PATA master)
Timing buffered disk reads: 114 MB in 3.02 seconds = 37.70 MB/sec
Timing buffered disk reads: 114 MB in 3.00 seconds = 37.97 MB/sec
Timing buffered disk reads: 116 MB in 3.05 seconds = 38.06 MB/sec
Timing buffered disk reads: 116 MB in 3.05 seconds = 38.07 MB/sec
/dev/sdc: (PATA slave)
Timing buffered disk reads: 164 MB in 3.03 seconds = 54.21 MB/sec
Timing buffered disk reads: 166 MB in 3.02 seconds = 55.04 MB/sec
Timing buffered disk reads: 166 MB in 3.01 seconds = 55.10 MB/sec
Timing buffered disk reads: 158 MB in 3.01 seconds = 52.41 MB/sec
/dev/sdd: (SATA backup drive)
Timing buffered disk reads: 314 MB in 3.00 seconds = 104.55 MB/sec
Timing buffered disk reads: 312 MB in 3.01 seconds = 103.67 MB/sec
Timing buffered disk reads: 308 MB in 3.01 seconds = 102.34 MB/sec
Timing buffered disk reads: 314 MB in 3.00 seconds = 104.60 MB/sec
The system drive throughput varies a lot, depending on other I/O.
I
> You're running KDE4, guess you went for the default and use mysql for
> "app- office/akonadi-server".
>
> I switched to using "sqlite" for this due to issues getting it to work
> with mysql. I think this might help there?
So I only have to set the sqlite use flag and remove the mysql use flag
for akonadi-server? I'm doing this now. And this also gives an example for
what is going on here:
I have just logged into KDE. I did not log out since yesterday, and when
started a VM with vmplayer, the system swapped like crazy, I could not use
it for minutes. After this, the panel did not react any more, and the
desktop background did not redraw, so I logged out and in again. The VM
started fine now. Well, could be faster, but maybe it was okay.
Then I started answering your mail, and tried to reemerge akonadi-server,
but I had a type, so portage took a long search for akomadi-server.
meanwhile the dektop became quite unresponsive, load went high, and I made
a screenshot [*]. If you look at the top right, gkrellm shows this above
'Proc'. The first increase at the left was after I started emerge, the 2nd
at the right was after I pressed the PrtSc key.
> > Performance does not feel too bad at first. But after a while, I
> > cannot even play videos during emerges. The playback stutters,
> > sometimes I have pauses for several seconds. As long as there is no
> > swap space occpied, it's not so bad I think. Maybe I have a probelm
> > with disk I/O, and things get much worse when swapping occurs. When
> > I look at iotop, I see various programs like chromium and various
> > KDE applications appear. I guess that's normal, but should not be
> > noticeable. Hey, there were times when I created a 2G tmpfs for
> > /var/tmp/portage, with only 3G on my 32bit system. BTW, I lowered my
> > swappiness to 10. This helps a little I think, because the swapping
> > occurs later, the system is more responsive.
>
> Do you also encrypt swap?
Yes.
> Disk I/O is, in my experience, a very common cause for "freeze-ups".
> Can you test with unencrypted disks to see if the issue occurs then as
> well?
Yes, I can do this. It's some work, but I tried so much, why not this. I
have some free space, and already have written a backup script that
automatically creates LVM snapshots, decrypts them, and backs it up, so I
can do this from the running system.
> > And I have similar problems when copying data between some old PATA
> > drives. When I copy stuff and do a mkfs on another partition, mplayer
> > sometimes stutters and hangs for ten seconds. No joy. Working with
> > KDE sucks, switching dektops sometimes takes ages, and even now I am
> > typing faster than kmail can display the characters. That's with am
> > emerge of chromium running, with PORTAGE_NICENESS=10 and using
> > ionice -c 3. Load is around 8, but sometimes gets even higher. And
> > then, load suddenly drops back to lower values, as if somthing was
> > blocking. Some applications swapping, maybe.
>
> Very possibly, maybe an idea to check which applications are hogging
> the memory. If it is the swapping of the system, then this will be
> caused by the most memory-hungry processes.
>
> Can you post the result of: "ps axu"?
> This will give an indication which processes are running and using a
> lot of memory.
First, here is free -m:
total used free shared buffers cached
Mem: 3452 3225 226 0 54 325
-/+ buffers/cache: 2844 607
Swap: 4094 935 3159
And here the output of top, sorted by memory. I think it is a similar
output to ps axu, but more condensed and better readable via mail. The
full ps aux output, sorted by memory, is in [2].
top - 21:23:03 up 1 day, 7 min, 11 users, load average: 0.04, 0.05, 0.02
Tasks: 418 total, 1 running, 417 sleeping, 0 stopped, 0 zombie
Cpu(s): 17.8%us, 12.7%sy, 9.9%ni, 45.9%id, 13.5%wa, 0.1%hi, 0.1%si, 0.0%st
Mem: 3534936k total, 3312312k used, 222624k free, 56264k buffers
Swap: 4193272k total, 957908k used, 3235364k free, 334048k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25993 root 20 0 847m 475m 47m S 21 13.8 49:01.11 X
26553 wonko 20 0 1312m 235m 10m S 0 6.8 0:28.32 java
26925 wonko 20 0 698m 112m 18m S 0 3.2 3:30.75 kontact
26961 wonko 20 0 526m 79m 13m S 0 2.3 2:16.01 chrome
26566 wonko 20 0 1128m 72m 10m S 0 2.1 1:02.46 amarok
27056 wonko 20 0 871m 57m 19m S 0 1.7 0:05.07 chrome
30324 root 30 10 195m 55m 1136 S 0 1.6 0:09.46 emerge
27051 wonko 20 0 874m 50m 8668 S 0 1.5 0:14.16 chrome
26126 wonko 20 0 1028m 44m 11m S 2 1.3 9:18.94 plasma-desktop
27046 wonko 20 0 878m 41m 7324 S 0 1.2 0:16.59 chrome
27122 wonko 20 0 871m 36m 7660 S 0 1.0 0:40.03 chrome
27036 wonko 20 0 865m 29m 7492 S 0 0.8 0:00.94 chrome
27198 wonko 20 0 860m 28m 6240 S 0 0.8 0:04.76 chrome
26101 wonko 20 0 427m 28m 8672 S 4 0.8 5:27.48 kwin
26298 wonko 20 0 396m 26m 9.9m S 0 0.8 0:02.91 knotes
27766 wonko 20 0 857m 26m 5924 S 0 0.8 0:03.03 chrome
26903 root 20 0 64316 25m 288 S 0 0.7 0:00.33 screen
27203 wonko 20 0 865m 24m 5960 S 0 0.7 0:04.06 chrome
30226 wonko 20 0 367m 23m 9232 S 0 0.7 0:03.21 gwenview
26221 wonko 20 0 609m 23m 3752 S 0 0.7 0:02.55 knotify4
X takes an awful lot, then comes java, which is running only for my
tvbrowser. And many many chrome processes.
> > Now I am out of ideas. I really hope someone here has one. I cannot
> > work with this system any more when emerges are going on.
>
> Had similar issues with a desktop machine myself, managed to kill some
> "features" that I wasn't using and this solved most of the problems.
I hope I can say this soon, too.
> Lets see where checking for IO-speeds and memory-usage of your apps
> take us :)
Thanks for your help! I appreciate this very much.
Wonko
[1] http://www.wonkology.org/comp/desktop/2010-08-25_emerge-akomadi.png
[2] http://www.wonkology.org/gentoo/
