RE: Solr 4.0 Optimize query very slow before the optimize end of a few minutes
Yes, I used the same query url for each curl-call, it is very simple "http://...q=OS01W:sina*&fl=SecId,OS01W&rows=1&wt=xml&indent=true";. -Original Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: 2013年6月14日 16:20 To: solr-user@lucene.apache.org Subject: RE: Solr 4.0 Optimize query very slow before the optimize end of a few minutes On Fri, 2013-06-14 at 06:59 +0200, Jeffery Wang wrote: > Time queryTime(ms), CPU % r/s w/s rMB/s wMB/s IO % > ... > 7:30:52 16594 26 36 0 0.140 99.3 > 7:30:53 31 80 368 0 42.43 0 94.3 > 7:31:23 28575 41 35 21 0.372.3695.9 > 7:32:22 53399 31 81 39 0.742.6399.5!!! > 7:32:23 11 54 155 0 16.46 0 99.6 > 7:33:28 60199 28 30 2 0.120.0199.8!! Having a single query that is slow is expected behaviour as the reader will have opened the merged segment and caches needs to be filled. But I do not know why you have more than one query that is slow. Do you use the same query for each curl-call? - Toke Eskildsen, State and University Library, Denmark
RE: Solr 4.0 Optimize query very slow before the optimize end of a few minutes
On Fri, 2013-06-14 at 06:59 +0200, Jeffery Wang wrote: > Time queryTime(ms), CPU % r/s w/s rMB/s wMB/s IO % > ... > 7:30:52 16594 26 36 0 0.140 99.3 > 7:30:53 31 80 368 0 42.43 0 94.3 > 7:31:23 28575 41 35 21 0.372.3695.9 > 7:32:22 53399 31 81 39 0.742.6399.5!!! > 7:32:23 11 54 155 0 16.46 0 99.6 > 7:33:28 60199 28 30 2 0.120.0199.8!! Having a single query that is slow is expected behaviour as the reader will have opened the merged segment and caches needs to be filled. But I do not know why you have more than one query that is slow. Do you use the same query for each curl-call? - Toke Eskildsen, State and University Library, Denmark
RE: Solr 4.0 Optimize query very slow before the optimize end of a few minutes
Hi Otis, Sorry, it does not formatted. TimequeryTime(ms), CPU % r/s w/s rMB/s wMB/s IO % ... 7:30:24 12 89 156.44 0 16.40 94.06 7:30:25 18 91 157 0 15.35 0 98.1 7:30:26 9 91 194 0 19.62 0 96.1 7:30:27 14 38 352 0 38.17 0 100.1 7:30:28 30 77 205.94 16.83 20.17 4.0298.51 7:30:30 101 88 396 0 45.99 0 90.7 7:30:31 11 90 120 0 11.34 0 97.5 7:30:32 38 89 262.38 0 28.03 0 96.24 7:30:33 11 78 68 17 4.894.9399.9 7:30:34 9 29 201 0 20.16 0 100.3 7:30:35 9 87 181 0 17.27 0 94.3 7:30:52 16594 26 36 0 0.140 99.3 7:30:53 31 80 368 0 42.43 0 94.3 7:31:23 28575 41 35 21 0.372.3695.9 7:31:27 267660 127 0 13.76 0 83.5 7:31:28 8 59 279 0 30.99 0 99.4 7:32:22 53399 31 81 39 0.742.6399.5!!! 7:32:23 11 54 155 0 16.46 0 99.6 7:32:24 9 47 63.37 4.954.180.0298.42 7:32:25 9 25 34 0 0.130 98.8 7:32:26 8 27 30 0 0.120 99.9 7:33:28 60199 28 30 2 0.120.0199.8!! But why it always query slow at the last few minutes. I have tested it many times the optimize will last for 2 hours , almost every time, the query is quick enough(query cost about 30ms) in the 2 hours, only slow at the last few minutes(query will cost 6ms). Thanks, Jeffery -Original Message- From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] Sent: 2013年6月14日 12:20 To: solr-user@lucene.apache.org Subject: Re: Solr 4.0 Optimize query very slow before the optimize end of a few minutes Hi, What you pasted from console didn't come across well. Yes, optimizing a static index is OK and yes, if your index is "very unoptimized" then yes, it will be slower than when it is optimized not sure if that addresses your concerns... Otis -- Solr & ElasticSearch Support -- http://sematext.com/ On Fri, Jun 14, 2013 at 12:04 AM, Jeffery Wang wrote: > Do someone known Why the query is very slow before the optimize end of a few > minutes. > > When the solr optimize, I have a loop query( curl "query url" and sleep one > second) every one second to check the query speed. It is normal, the query > time can be accept. But it always very slow before the optimize end of a few > minutes. > The solr index size is about 22G after optimized. > > The follows is the query time cost, CPU and IO usage. The whole optimize > process, the IO is high, it can be understand. > time > > query time(ms) > > CPU % > > r/s > > w/s > > rMB/s > > wMB/s > > IO % > > 7:30:24 > > 12 > > 89 > > 156.44 > > 0 > > 16.4 > > 0 > > 94.06 > > 7:30:25 > > 18 > > 91 > > 157 > > 0 > > 15.35 > > 0 > > 98.1 > > 7:30:26 > > 9 > > 91 > > 194 > > 0 > > 19.62 > > 0 > > 96.1 > > 7:30:27 > > 14 > > 38 > > 352 > > 0 > > 38.17 > > 0 > > 100.1 > > 7:30:28 > > 30 > > 77 > > 205.94 > > 16.83 > > 20.17 > > 4.02 > > 98.51 > > 7:30:30 > > 101 > > 88 > > 396 > > 0 > > 45.99 > > 0 > > 90.7 > > 7:30:31 > > 11 > > 90 > > 120 > > 0 > > 11.34 > > 0 > > 97.5 > > 7:30:32 > > 38 > > 89 > > 262.38 > > 0 > > 28.03 > > 0 > > 96.24 > > 7:30:33 > > 11 > > 78 > > 68 > > 17 > > 4.89 > > 4.93 > > 99.9 > > 7:30:34 > > 9 > > 29 > > 201 > > 0 > > 20.16 > > 0 > > 100.3 > > 7:30:35 > > 9 > > 87 > > 181 > > 0 > > 17.27 > > 0 > > 94.3 > > 7:30:52 > > 16594 > > 26 > > 36 > > 0 > > 0.14 > > 0 > > 99.3 > > 7:30:53 > > 31 > > 80 > > 368 > > 0 > > 42.43 > > 0 > > 94.3 > > 7:31:23 > > 28575 > > 41 > > 35 > > 21 > > 0.37 > > 2.36 > > 95.9 > > 7:31:27 > > 2676 > > 60 > > 127 > > 0 > > 13.76 > > 0 > > 83.5 > > 7:31:28 > > 8 > > 59 > > 279 > > 0 > > 30.99 > > 0 > > 99.4 > > 7:32:22 > > 53399 > > 31 > > 81 > > 39 > > 0.74 > > 2.63 > > 99.5 > > 7:32:23 > > 11 > > 54 > > 155 > > 0 > > 16.46 > > 0 > > 99.6 > > 7:32:24 > > 9 > > 47 > > 63.37 > > 4.95 > > 4.18 > > 0.02 > > 98.42 > > 7:32:25 > > 9 > > 25 > > 34 > > 0 > > 0.13 > > 0 > > 98.8 > > 7:32:26 > > 8 > > 27 > > 30 > > 0 > > 0.12 > > 0 > > 99.9 > > 7:33:28 > > 60199 > > 28 > > 30 > > 2 > > 0.12 > > 0.01 > > 99.8 > > > Thanks, > __ > > Jeffery Wang
Re: Solr 4.0 Optimize query very slow before the optimize end of a few minutes
Hi, What you pasted from console didn't come across well. Yes, optimizing a static index is OK and yes, if your index is "very unoptimized" then yes, it will be slower than when it is optimized not sure if that addresses your concerns... Otis -- Solr & ElasticSearch Support -- http://sematext.com/ On Fri, Jun 14, 2013 at 12:04 AM, Jeffery Wang wrote: > Do someone known Why the query is very slow before the optimize end of a few > minutes. > > When the solr optimize, I have a loop query( curl "query url" and sleep one > second) every one second to check the query speed. It is normal, the query > time can be accept. But it always very slow before the optimize end of a few > minutes. > The solr index size is about 22G after optimized. > > The follows is the query time cost, CPU and IO usage. The whole optimize > process, the IO is high, it can be understand. > time > > query time(ms) > > CPU % > > r/s > > w/s > > rMB/s > > wMB/s > > IO % > > 7:30:24 > > 12 > > 89 > > 156.44 > > 0 > > 16.4 > > 0 > > 94.06 > > 7:30:25 > > 18 > > 91 > > 157 > > 0 > > 15.35 > > 0 > > 98.1 > > 7:30:26 > > 9 > > 91 > > 194 > > 0 > > 19.62 > > 0 > > 96.1 > > 7:30:27 > > 14 > > 38 > > 352 > > 0 > > 38.17 > > 0 > > 100.1 > > 7:30:28 > > 30 > > 77 > > 205.94 > > 16.83 > > 20.17 > > 4.02 > > 98.51 > > 7:30:30 > > 101 > > 88 > > 396 > > 0 > > 45.99 > > 0 > > 90.7 > > 7:30:31 > > 11 > > 90 > > 120 > > 0 > > 11.34 > > 0 > > 97.5 > > 7:30:32 > > 38 > > 89 > > 262.38 > > 0 > > 28.03 > > 0 > > 96.24 > > 7:30:33 > > 11 > > 78 > > 68 > > 17 > > 4.89 > > 4.93 > > 99.9 > > 7:30:34 > > 9 > > 29 > > 201 > > 0 > > 20.16 > > 0 > > 100.3 > > 7:30:35 > > 9 > > 87 > > 181 > > 0 > > 17.27 > > 0 > > 94.3 > > 7:30:52 > > 16594 > > 26 > > 36 > > 0 > > 0.14 > > 0 > > 99.3 > > 7:30:53 > > 31 > > 80 > > 368 > > 0 > > 42.43 > > 0 > > 94.3 > > 7:31:23 > > 28575 > > 41 > > 35 > > 21 > > 0.37 > > 2.36 > > 95.9 > > 7:31:27 > > 2676 > > 60 > > 127 > > 0 > > 13.76 > > 0 > > 83.5 > > 7:31:28 > > 8 > > 59 > > 279 > > 0 > > 30.99 > > 0 > > 99.4 > > 7:32:22 > > 53399 > > 31 > > 81 > > 39 > > 0.74 > > 2.63 > > 99.5 > > 7:32:23 > > 11 > > 54 > > 155 > > 0 > > 16.46 > > 0 > > 99.6 > > 7:32:24 > > 9 > > 47 > > 63.37 > > 4.95 > > 4.18 > > 0.02 > > 98.42 > > 7:32:25 > > 9 > > 25 > > 34 > > 0 > > 0.13 > > 0 > > 98.8 > > 7:32:26 > > 8 > > 27 > > 30 > > 0 > > 0.12 > > 0 > > 99.9 > > 7:33:28 > > 60199 > > 28 > > 30 > > 2 > > 0.12 > > 0.01 > > 99.8 > > > Thanks, > __ > Jeffery Wang
Solr 4.0 Optimize query very slow before the optimize end of a few minutes
Do someone known Why the query is very slow before the optimize end of a few minutes. When the solr optimize, I have a loop query( curl "query url" and sleep one second) every one second to check the query speed. It is normal, the query time can be accept. But it always very slow before the optimize end of a few minutes. The solr index size is about 22G after optimized. The follows is the query time cost, CPU and IO usage. The whole optimize process, the IO is high, it can be understand. time query time(ms) CPU % r/s w/s rMB/s wMB/s IO % 7:30:24 12 89 156.44 0 16.4 0 94.06 7:30:25 18 91 157 0 15.35 0 98.1 7:30:26 9 91 194 0 19.62 0 96.1 7:30:27 14 38 352 0 38.17 0 100.1 7:30:28 30 77 205.94 16.83 20.17 4.02 98.51 7:30:30 101 88 396 0 45.99 0 90.7 7:30:31 11 90 120 0 11.34 0 97.5 7:30:32 38 89 262.38 0 28.03 0 96.24 7:30:33 11 78 68 17 4.89 4.93 99.9 7:30:34 9 29 201 0 20.16 0 100.3 7:30:35 9 87 181 0 17.27 0 94.3 7:30:52 16594 26 36 0 0.14 0 99.3 7:30:53 31 80 368 0 42.43 0 94.3 7:31:23 28575 41 35 21 0.37 2.36 95.9 7:31:27 2676 60 127 0 13.76 0 83.5 7:31:28 8 59 279 0 30.99 0 99.4 7:32:22 53399 31 81 39 0.74 2.63 99.5 7:32:23 11 54 155 0 16.46 0 99.6 7:32:24 9 47 63.37 4.95 4.18 0.02 98.42 7:32:25 9 25 34 0 0.13 0 98.8 7:32:26 8 27 30 0 0.12 0 99.9 7:33:28 60199 28 30 2 0.12 0.01 99.8 Thanks, __ Jeffery Wang
