John, thanks again for taking the time to think about this. I'm starting to
think it's a slicehost problem.
Over the past couple of days I've tried:
1. increased memory from 256 to 512MB on the slice
- lots of free memory, minor performance boost
2. ran sqlite3 in production
- no noticeable difference in DB access times. not so surprising,
lightly used system
3. tuned mysql
- followed settings from an example config for system with 512MB,
mysql + webserver
- some improvement
4. ran nginx+passenger with 2 worker processes
- access times were about the same as litespeed
5. verified that all html was valid
6. searched system logs for any signs of problems
- no indication of issues
7. updated all gems to latest release (backing off rack 1.2.2 to 1.1.0)
- this included the latest radiant 0.9.0
- mysql gem is at 2.8.1
8. rake radiant:unfreeze -- running from the gem instead of edge
9. made sure all extensions were up to date (git pull'ed them all)
- database migrations, extension migrations all okay
None of this has made any difference in the total access times -- still very
high.
(sample mysql+litespeed)
Completed in 442ms (DB: 5) | 200 OK [
http://octopusgardenyoga.com/css/octopus]
Completed in 216ms (DB: 5) | 200 OK [
http://octopusgardenyoga.com/css/gallery]
Completed in 114ms (DB: 6) | 200 OK [http://octopusgardenyoga.com/css/print]
Completed in 3163ms (DB: 187) | 200 [
http://www.octopusgardenyoga.com/events/]
Completed in 1687ms (DB: 78) | 200 [http://www.octopusgardenyoga.com/]
Completed in 392ms (DB: 10) | 200 OK [
http://www.octopusgardenyoga.com/css/gallery]
Completed in 875ms (DB: 47) | 200 OK [
http://www.octopusgardenyoga.com/css/octopus]
Completed in 260ms (DB: 18) | 200 OK [
http://www.octopusgardenyoga.com/css/print]
Completed in 2467ms (DB: 83) | 200 [http://octopusgardenyoga.com/]
Completed in 195ms (DB: 10) | 200 OK [http://octopusgardenyoga.com/css/print
]
Completed in 374ms (DB: 11) | 200 OK [
http://octopusgardenyoga.com/css/gallery]
Completed in 634ms (DB: 130) | 200 OK [
http://octopusgardenyoga.com/css/octopus]
Completed in 1494ms (DB: 81) | 200 [http://octopusgardenyoga.com/fees/]
Completed in 1525ms (DB: 169) | 200 [
http://octopusgardenyoga.com/guidelines/]
Completed in 1216ms (DB: 37) | 200 [http://octopusgardenyoga.com/schedule/]
(sample mysql+nginx/passenger)
Completed in 1328ms (DB: 153) | 200 [
http://www.octopusgardenyoga.com/classes/]
Completed in 1050ms (DB: 60) | 200 [
http://www.octopusgardenyoga.com/FAQ/how-do-i-get-started/]
Completed in 1259ms (DB: 64) | 200 [http://octopusgardenyoga.com/teachers/]
Completed in 263ms (DB: 69) | 200 OK [http://octopusgardenyoga.com/css/print
]
Completed in 364ms (DB: 10) | 200 OK [
http://octopusgardenyoga.com/css/gallery]
Completed in 609ms (DB: 8) | 200 OK [
http://octopusgardenyoga.com/css/octopus]
Completed in 2378ms (DB: 107) | 200 [http://octopusgardenyoga.com/events]
Completed in 1164ms (DB: 34) | 200 [http://octopusgardenyoga.com/news]
Completed in 1571ms (DB: 214) | 200 [http://octopusgardenyoga.com/photos]
Completed in 2441ms (DB: 360) | 200 [
http://octopusgardenyoga.com/photos/mexico-2006]
Completed in 1328ms (DB: 56) | 200 [http://octopusgardenyoga.com/]
Completed in 1143ms (DB: 50) | 200 [
http://www.octopusgardenyoga.com/schedule/]
Completed in 248ms (DB: 5) | 200 OK [
http://www.octopusgardenyoga.com/css/gallery]
Completed in 281ms (DB: 131) | 200 OK [
http://www.octopusgardenyoga.com/css/print]
Completed in 572ms (DB: 11) | 200 OK [
http://www.octopusgardenyoga.com/css/octopus]
Completed in 2112ms (DB: 119) | 200 [
http://octopusgardenyoga.com/photos/mexico-2006]
Completed in 3140ms (DB: 418) | 200 [
http://octopusgardenyoga.com/photos/mexico-2008]
This morning I could not ssh into the slice. Used the Slicehost console and
found this message being repeated continuously:
Out of Memory: Killed process XXXX (ruby)
Had to do a hard reboot to get the system back. I've just started searching
the slicehost forums, found a few others reporting similar problems, so
maybe I'll find something here.
Is it unusual to have 4 ruby processes using about 20% of available memory
each? This is what I'm seeing on top: (BTW, the 0 swap seems to be a result
of resizing the slice. This has been reported on the forums and I'm about to
submit a ticket, but with lots of available memory, it shouldn't be an
immediate problem).
top - 13:06:37 up 2:05, 1 user, load average: 0.00, 0.02, 0.00
Tasks: 65 total, 1 running, 64 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.2%id, 0.8%wa, 0.0%hi, 0.0%si,
0.0%st
Mem: 524460k total, 370640k used, 153820k free, 28396k buffers
Swap: 0k total, 0k used, 0k free, 65716k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2630 marshal 15 0 218m 108m 4416 S 0 21.2 0:05.27 ruby
3388 marshal 15 0 219m 106m 2112 S 0 20.9 0:03.13 ruby
3389 marshal 15 0 218m 106m 1784 S 0 20.7 0:00.47 ruby
2394 mysql 15 0 206m 22m 5572 S 0 4.4 0:03.77 mysqld
2695 marshal 15 0 20040 4064 1364 S 0 0.8 0:00.10 bash
2744 root 15 0 19524 3808 1372 S 0 0.7 0:00.14 bash
2692 root 16 0 53468 2708 2140 S 0 0.5 0:00.02 sshd
2628 www-data 15 0 6340 2308 1328 S 0 0.4 0:00.18 lshttpd
2545 root 16 0 36672 2124 1680 S 0 0.4 0:00.00 master
2549 postfix 15 0 38772 2080 1652 S 0 0.4 0:00.01 qmgr
Thanks again for looking at this. Much appreciated.
--
marshal
