Hi all,
This is from the site running on 3.3+ad13 that keeps hanging. I can't upgrade it until we're positive nsopenssl is ok, so I'm stuck debugging this for the time being.
I created a scheduled proc that logs "ns_server threads" and "ns_server all" every 30 seconds. This is what I get as the number of idle threads starts to drop, culminating in zero idle threads and queueing beginning at the last snapshot. I didn't continue cutting and pasting but the situation gets worse and worse until the keepalive module kicks in and restarts the server.
The product-thumbnail.jpg files are on the home page, but most of the time it works fine. This is not a busy site and there's no good reason for it to be using up 30 threads at a time; something has to be wrong when that happens. It looks like new threads are able to get started, and they are accruing time, but they can't actually be running properly as none of those pages takes the many seconds shown here to complete (the next to the last number is the number of seconds the connection has been alive, and the last number is the bytes transferred).
When this happens there is almost no load on the system, because nothing is actually running. None of the GETs shown here writes anything to the database so there shouldn't be any deadlock type problems.
What I need to do now is figure out what other data I can collect that will tell me why threads suddenly start to stack up. Any suggestions? I'm wondering about scheduled procs, since they don't seem to show up here; is there some way to tell which of those is currently running when I take my snapshot?
thanks,
janine
5:37:16 am:
{74036 210.86.34.47 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 26 80000} {74037
210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 26 0} {74038
210.86.34.47 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 26 64000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 25 0} {74040
217.154.9.101 running GET /back-issues/ 23 0}
{74041 194.126.101.134 running GET /extracts/659 14 0}5:37:46 am:
{74036 210.86.34.47 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 56 144000} {74037
210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 56 0} {74038
210.86.34.47 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 56 112000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 55 0} {74040
217.154.9.101 running GET /back-issues/ 53 0}
{74041 194.126.101.134 running GET /extracts/659 44 0}
{74042 217.154.9.101 running GET /back-issues/ 29 0}
{74043 62.49.253.50 running GET / 23 0}
{74044 66.196.90.112 running GET /article-excerpt 14 0}
{74045 61.3.200.208 running GET /topsellers 13 0}5:38:16 am:
{74036 210.86.34.47 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 86 192000} {74037
210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 86 0} {74038
210.86.34.47 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 86 160000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 85 0} {74040
217.154.9.101 running GET /back-issues/ 83 0}
{74041 194.126.101.134 running GET /extracts/659 74 0}
{74042 217.154.9.101 running GET /back-issues/ 59 0}
{74043 62.49.253.50 running GET / 53 0}
{74044 66.196.90.112 running GET /article-excerpt 44 0}
{74045 61.3.200.208 running GET /topsellers 43 0}
{74046 212.70.164.106 running GET / 19 0}5:38:46 am:
{74036 210.86.34.47 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 116 240000} {74037
210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 116 0} {74038
210.86.34.47 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 116 224000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 115 0} {74040
217.154.9.101 running GET /back-issues/ 113 0}
{74041 194.126.101.134 running GET /extracts/659 104 0}
{74042 217.154.9.101 running GET /back-issues/ 89 0}
{74043 62.49.253.50 running GET / 83 0}
{74044 66.196.90.112 running GET /article-excerpt 74 0}
{74045 61.3.200.208 running GET /topsellers 73 0}
{74046 212.70.164.106 running GET / 49 0}
{74047 217.154.9.101 running GET /latest-issue 18 0}
{74048 65.214.36.186 running GET /authors/960 17 0}
{74049 217.154.9.101 running GET /back-issues/ 15 0}
{74050 64.68.82.47 running GET /authors/189 12 0}
{74051 217.154.9.101 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 10 0} {74052
217.154.9.101 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 10 0} {74053
217.154.9.101 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 10 0} {74054
24.11.110.108 running GET /latest-issue 7 0}5:39:16 am:
{74036 210.86.34.47 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 146 304000} {74037
210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 146 0} {74038
210.86.34.47 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 146 288000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 145 0} {74040
217.154.9.101 running GET /back-issues/ 143 0}
{74041 194.126.101.134 running GET /extracts/659 134 0}
{74042 217.154.9.101 running GET /back-issues/ 119 0}
{74043 62.49.253.50 running GET / 113 0}
{74044 66.196.90.112 running GET /article-excerpt 104 0}
{74045 61.3.200.208 running GET /topsellers 103 0}
{74046 212.70.164.106 running GET / 79 0}
{74047 217.154.9.101 running GET /latest-issue 48 0}
{74048 65.214.36.186 running GET /authors/960 47 0}
{74049 217.154.9.101 running GET /back-issues/ 45 0}
{74050 64.68.82.47 running GET /authors/189 42 0}
{74051 217.154.9.101 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 40 0} {74052
217.154.9.101 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 40 0} {74053
217.154.9.101 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 40 0} {74054
24.11.110.108 running GET /latest-issue 37 0}
{74055 209.237.238.181 running GET /extracts/2144 15 0}
{74056 131.107.76.139 running GET /robots.txt 15 0}
{74057 64.68.82.159 running GET /authors/2324 5 0}
{74058 207.158.50.105 running GET / 4 0}
{74059 209.202.133.59 running GET /SYSTEM/dbtest.tcl 2 0}5:39:46 am:
{74036 210.86.34.47 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 176 352000} {74037
210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 176 0} {74038
210.86.34.47 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 176 336000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 175 0} {74040
217.154.9.101 running GET /back-issues/ 173 0}
{74041 194.126.101.134 running GET /extracts/659 164 0}
{74042 217.154.9.101 running GET /back-issues/ 149 0}
{74043 62.49.253.50 running GET / 143 0}
{74044 66.196.90.112 running GET /article-excerpt 134 0}
{74045 61.3.200.208 running GET /topsellers 133 0}
{74046 212.70.164.106 running GET / 109 0}
{74047 217.154.9.101 running GET /latest-issue 78 0}
{74048 65.214.36.186 running GET /authors/960 77 0}
{74049 217.154.9.101 running GET /back-issues/ 75 0}
{74050 64.68.82.47 running GET /authors/189 72 0}
{74051 217.154.9.101 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 70 0} {74052
217.154.9.101 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 70 0} {74053
217.154.9.101 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 70 0} {74054
24.11.110.108 running GET /latest-issue 67 0}
{74055 209.237.238.181 running GET /extracts/2144 45 0}
{74056 131.107.76.139 running GET /robots.txt 45 0}
{74057 64.68.82.159 running GET /authors/2324 35 0}
{74058 207.158.50.105 running GET / 34 0}
{74059 209.202.133.59 running GET /SYSTEM/dbtest.tcl 32 0}
{74060 80.58.33.172 running GET /shop/product 21 0}
{74061 209.237.238.181 running GET /extracts/2144 15 0}
{74062 66.196.90.27 running GET /shop/product 5 0}5:40:18 am:
{74037 210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 208 64000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 207 16000} {74040
217.154.9.101 running GET /back-issues/ 205 0}
{74041 194.126.101.134 running GET /extracts/659 196 0}
{74042 217.154.9.101 running GET /back-issues/ 181 0}
{74043 62.49.253.50 running GET / 175 0}
{74044 66.196.90.112 running GET /article-excerpt 166 0}
{74045 61.3.200.208 running GET /topsellers 165 0}
{74046 212.70.164.106 running GET / 141 0}
{74047 217.154.9.101 running GET /latest-issue 110 0}
{74048 65.214.36.186 running GET /authors/960 109 0}
{74049 217.154.9.101 running GET /back-issues/ 107 0}
{74050 64.68.82.47 running GET /authors/189 104 0}
{74051 217.154.9.101 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 102 0} {74052
217.154.9.101 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 102 0} {74053
217.154.9.101 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 102 0} {74054
24.11.110.108 running GET /latest-issue 99 0}
{74055 209.237.238.181 running GET /extracts/2144 77 0}
{74056 131.107.76.139 running GET /robots.txt 77 0}
{74057 64.68.82.159 running GET /authors/2324 67 0}
{74058 207.158.50.105 running GET / 66 0}
{74059 209.202.133.59 running GET /SYSTEM/dbtest.tcl 64 0}
{74060 80.58.33.172 running GET /shop/product 53 0}
{74061 209.237.238.181 running GET /extracts/2144 47 0}
{74062 66.196.90.27 running GET /shop/product 37 0}
{74063 210.86.34.47 running GET
/shop/product-file/99/driv1999/product-thumbnail.jpg 18 0} {74064
210.86.34.47 running GET
/shop/product-file/13/mone1513/product-thumbnail.jpg 9 0} {74065
209.237.238.181 running GET /extracts/2144 2 0}5:40:49 am:
{74037 210.86.34.47 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 239 128000} {74039
210.86.34.47 running GET
/shop/product-file/98/whos1998/product-thumbnail.jpg 238 96000} {74040
217.154.9.101 running GET /back-issues/ 236 0}
{74041 194.126.101.134 running GET /extracts/659 227 0}
{74042 217.154.9.101 running GET /back-issues/ 212 0}
{74043 62.49.253.50 running GET / 206 0}
{74044 66.196.90.112 running GET /article-excerpt 197 0}
{74045 61.3.200.208 running GET /topsellers 196 0}
{74046 212.70.164.106 running GET / 172 0}
{74047 217.154.9.101 running GET /latest-issue 141 0}
{74048 65.214.36.186 running GET /authors/960 140 0}
{74049 217.154.9.101 running GET /back-issues/ 138 0}
{74050 64.68.82.47 running GET /authors/189 135 0}
{74051 217.154.9.101 running GET
/shop/product-file/58/rowi1958/product-thumbnail.jpg 133 0} {74052
217.154.9.101 running GET
/shop/product-file/60/curi1960/product-thumbnail.jpg 133 0} {74053
217.154.9.101 running GET
/shop/product-file/61/thel1961/product-thumbnail.jpg 133 0} {74054
24.11.110.108 running GET /latest-issue 130 0}
{74055 209.237.238.181 running GET /extracts/2144 108 0}
{74056 131.107.76.139 running GET /robots.txt 108 0}
{74057 64.68.82.159 running GET /authors/2324 98 0}
{74058 207.158.50.105 running GET / 97 0}
{74059 209.202.133.59 running GET /SYSTEM/dbtest.tcl 95 0}
{74060 80.58.33.172 running GET /shop/product 84 0}
{74061 209.237.238.181 running GET /extracts/2144 78 0}
{74062 66.196.90.27 running GET /shop/product 68 0}
{74063 210.86.34.47 running GET
/shop/product-file/99/driv1999/product-thumbnail.jpg 49 0} {74064
210.86.34.47 running GET
/shop/product-file/13/mone1513/product-thumbnail.jpg 40 0} {74065
209.237.238.181 running GET /extracts/2144 33 0}
{74066 207.158.50.105 running GET / 21 0}
{74067 61.1.234.195 running GET /extracts/1222 20 0}
{74068 64.68.82.185 queued ? ? 13 0}
{74069 212.27.41.32 queued ? ? 3 0}
{74070 217.42.43.35 queued ? ? 3 0}
-- AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
