Re: Solr - Load Increasing.

2009-11-20 Thread kalidoss

Thank u all.

   I have increased the heap size memory from 1gb to 1.5gb. Now its 
java -Xms512M -Xmx1536M -jar start.jar, My cpu load is normal and solr 
is not restating frequently,


   My autocommit maxdoc increased to 200.

   For last 24 hours no issue on load/restarts.

Thanks Guys.
Kalidoss.m,

Otis Gospodnetic wrote:

Your autocommit settings are still pretty aggressive causing very frequent 
commits, and that is using your CPU.
Yes, splitting the servers into a master and slaves tends to be the 
performant/scalable way to go.  There is no real downside to replication, 
really, just a bit of network traffic.

 Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



- Original Message 

From: kalidoss kalidoss.muthuramalin...@sifycorp.com
To: solr-user@lucene.apache.org
Sent: Wed, November 18, 2009 2:25:05 AM
Subject: Re: Solr - Load Increasing.

There seems to be some improvement. The writes speeds are faster. Server 
restarts are lower.


We changed the configuration to:
50
1

Before the Change:
- Server Restarts: 10 times in 12 hours
- CPU load: Average:50 and Peak:90

After the Change:
- Server Restarts: 4 times in 12 hours.
- CPU load: Average:30 and Peak:~70

Our every day writes are around 60k and reads are around 1 million.

We are now changing the MaxDocs to 300 and MaxTime will be 1 ms and 
hoping to some more improvements.


The system configuration is 4GB RAM and 4 core x 2 CPUs. We start the
solr (1.3) like this: java -Xms512M -Xmx1024M -jar start.jar

Is there any other way we can reduce the high CPU load in the system?

Do you guys think that upgrading to 1.4 and having the replication in 
place with reads and writes split into separate solrs will help? How 
efficient will the replication be with above mentioned scenarios? Is 
there any place we can look at for info on the disadvantages of 
replication...


Please help.
Kalidoss.m,
Tom Alt wrote:

Nice to learn a new word for the day!

But to answer your question, or at least part of it, I don't really think
you want a configuration like

 
 1

 10
 


Committing every doc, and every 10 milliseconds? That's just asking for
problems. How about starting with 1000 docs, and five minutes for maxTime
(5*60*1000) or about 3 laks of milliseconds.

That should help performance a lot. Try that, and see how it works.

Tom

On Mon, Nov 16, 2009 at 2:43 PM, Shashi Kant wrote:

 

I think it would be useful for members of this list to realize that not
everyone uses the same metrology and terms.

It is very easy for Americans to use the imperial system and presume
everyone does the same; Europeans to use the metric system etc. Hopefully
members on this list would be persuaded to use or at least clarify their
terminology.

While the apocryphal saying goes  the great thing about standards is they
are so many choose from, we should all make an effort to communicate
across
cultures and nations.



On Mon, Nov 16, 2009 at 5:33 PM, Israel Ekpo wrote:

   
On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood 
 

wrote:
   
Probably lakh: 100,000.


So, 900k qpd and 3M docs.

http://en.wikipedia.org/wiki/Lakh

wunder

On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:

   

Hi,

Your autoCommit settings are very aggressive.  I'm guessing that's
 

what's
 

causing the CPU load.
   

btw. what is laks?

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



- Original Message 
 
From: kalidoss 
To: solr-user@lucene.apache.org

Sent: Mon, November 16, 2009 9:11:21 AM
Subject: Solr - Load Increasing.

Hi All.

  My server solr box cpu utilization  increasing b/w 60 to 90% and
   

some
 

time
   

solr is getting down and we are restarting it manually.

  No of documents in solr 30 laks.
  No of add/update requrest solr 30 thousand / day. Avg of every 30
   

minutes
   

around 500 writes.
  No of search request 9laks / day.
  Size of the data directory: 4gb.


  My system ram is 8gb.
  System available space 12gb.
  processor Family: Pentium Pro

  Our solr data size can be increase in number like 90 laks. and
   

writes
 

per day
   

will be around 1laks.   - Hope its possible by solr.

  For write commit i have configured like

 1
 10


  Is all above can be possible? 90laks datas and 1laks per day
   

writes
   

and
   

30laks per day read??  - if yes what type of system configuration
   

would
 

require.
   

  Please suggest us.

thanks,
Kalidoss.m,


Get your world in your inbox!

Mail, widgets, documents, spreadsheets, organizer and much more with
   

your
   

Sifymail WIYI id!
Log on to http://www.sify.com

** DISCLAIMER **
Information contained and transmitted by this E-MAIL is proprietary

Re: Solr - Load Increasing.

2009-11-17 Thread kalidoss
There seems to be some improvement. The writes speeds are faster. Server 
restarts are lower.


We changed the configuration to:
maxDocs50/maxDocs
maxTime1/maxTime

Before the Change:
- Server Restarts: 10 times in 12 hours
- CPU load: Average:50 and Peak:90

After the Change:
- Server Restarts: 4 times in 12 hours.
- CPU load: Average:30 and Peak:~70

Our every day writes are around 60k and reads are around 1 million.

We are now changing the MaxDocs to 300 and MaxTime will be 1 ms and 
hoping to some more improvements.


The system configuration is 4GB RAM and 4 core x 2 CPUs. We start the
solr (1.3) like this: java -Xms512M -Xmx1024M -jar start.jar

Is there any other way we can reduce the high CPU load in the system?

Do you guys think that upgrading to 1.4 and having the replication in 
place with reads and writes split into separate solrs will help? How 
efficient will the replication be with above mentioned scenarios? Is 
there any place we can look at for info on the disadvantages of 
replication...


Please help.
Kalidoss.m,
Tom Alt wrote:

Nice to learn a new word for the day!

But to answer your question, or at least part of it, I don't really think
you want a configuration like

  autoCommit
  maxDocs1/maxDocs
  maxTime10/maxTime
  /autoCommit

Committing every doc, and every 10 milliseconds? That's just asking for
problems. How about starting with 1000 docs, and five minutes for maxTime
(5*60*1000) or about 3 laks of milliseconds.

That should help performance a lot. Try that, and see how it works.

Tom

On Mon, Nov 16, 2009 at 2:43 PM, Shashi Kant sk...@sloan.mit.edu wrote:

  

I think it would be useful for members of this list to realize that not
everyone uses the same metrology and terms.

It is very easy for Americans to use the imperial system and presume
everyone does the same; Europeans to use the metric system etc. Hopefully
members on this list would be persuaded to use or at least clarify their
terminology.

While the apocryphal saying goes  the great thing about standards is they
are so many choose from, we should all make an effort to communicate
across
cultures and nations.



On Mon, Nov 16, 2009 at 5:33 PM, Israel Ekpo israele...@gmail.com wrote:



On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood wun...@wunderwood.org
  

wrote:

Probably lakh: 100,000.


So, 900k qpd and 3M docs.

http://en.wikipedia.org/wiki/Lakh

wunder

On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:



Hi,

Your autoCommit settings are very aggressive.  I'm guessing that's
  

what's
  

causing the CPU load.


btw. what is laks?

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



- Original Message 
  

From: kalidoss kalidoss.muthuramalin...@sifycorp.com
To: solr-user@lucene.apache.org
Sent: Mon, November 16, 2009 9:11:21 AM
Subject: Solr - Load Increasing.

Hi All.

  My server solr box cpu utilization  increasing b/w 60 to 90% and


some
  

time


solr is getting down and we are restarting it manually.

  No of documents in solr 30 laks.
  No of add/update requrest solr 30 thousand / day. Avg of every 30


minutes


around 500 writes.
  No of search request 9laks / day.
  Size of the data directory: 4gb.


  My system ram is 8gb.
  System available space 12gb.
  processor Family: Pentium Pro

  Our solr data size can be increase in number like 90 laks. and


writes
  

per day


will be around 1laks.   - Hope its possible by solr.

  For write commit i have configured like

 1
 10


  Is all above can be possible? 90laks datas and 1laks per day


writes


and


30laks per day read??  - if yes what type of system configuration


would
  

require.


  Please suggest us.

thanks,
Kalidoss.m,


Get your world in your inbox!

Mail, widgets, documents, spreadsheets, organizer and much more with


your


Sifymail WIYI id!
Log on to http://www.sify.com

** DISCLAIMER **
Information contained and transmitted by this E-MAIL is proprietary


to


Sify


Limited and is intended for use only by the individual or entity to


which it is


addressed, and may contain information that is privileged,


confidential
  

or


exempt from disclosure under applicable law. If this is a forwarded


message, the


content of this E-MAIL may not have been sent with the authority of


the
  

Company.


If you are not the intended recipient, an agent of the intended


recipient or a


person responsible for delivering the information to the named


recipient,  you


are notified that any use, distribution, transmission, printing

Solr - Load Increasing.

2009-11-16 Thread kalidoss

Hi All.

   My server solr box cpu utilization  increasing b/w 60 to 90% and 
some time solr is getting down and we are restarting it manually.


   No of documents in solr 30 laks.
   No of add/update requrest solr 30 thousand / day. Avg of every 30 
minutes around 500 writes.

   No of search request 9laks / day.
   Size of the data directory: 4gb.


   My system ram is 8gb.
   System available space 12gb.
   processor Family: Pentium Pro

   Our solr data size can be increase in number like 90 laks. and 
writes per day will be around 1laks.   - Hope its possible by solr.


   For write commit i have configured like
   autoCommit
   maxDocs1/maxDocs
   maxTime10/maxTime
   /autoCommit

   Is all above can be possible? 90laks datas and 1laks per day writes 
and 30laks per day read??  - if yes what type of system configuration 
would require.


   Please suggest us.

thanks,
Kalidoss.m,
  



Get your world in your inbox!

Mail, widgets, documents, spreadsheets, organizer and much more with your 
Sifymail WIYI id!
Log on to http://www.sify.com

** DISCLAIMER **
Information contained and transmitted by this E-MAIL is proprietary to 
Sify Limited and is intended for use only by the individual or entity to 
which it is addressed, and may contain information that is privileged, 
confidential or exempt from disclosure under applicable law. If this is a 
forwarded message, the content of this E-MAIL may not have been sent with 
the authority of the Company. If you are not the intended recipient, an 
agent of the intended recipient or a  person responsible for delivering the 
information to the named recipient,  you are notified that any use, 
distribution, transmission, printing, copying or dissemination of this 
information in any way or in any manner is strictly prohibited. If you have 
received this communication in error, please delete this mail  notify us 
immediately at ad...@sifycorp.com


Re: Solr - Load Increasing.

2009-11-16 Thread Otis Gospodnetic
Hi,

Your autoCommit settings are very aggressive.  I'm guessing that's what's 
causing the CPU load.

btw. what is laks?

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



- Original Message 
 From: kalidoss kalidoss.muthuramalin...@sifycorp.com
 To: solr-user@lucene.apache.org
 Sent: Mon, November 16, 2009 9:11:21 AM
 Subject: Solr - Load Increasing. 
 
 Hi All.
 
My server solr box cpu utilization  increasing b/w 60 to 90% and some time 
 solr is getting down and we are restarting it manually.
 
No of documents in solr 30 laks.
No of add/update requrest solr 30 thousand / day. Avg of every 30 minutes 
 around 500 writes.
No of search request 9laks / day.
Size of the data directory: 4gb.
 
 
My system ram is 8gb.
System available space 12gb.
processor Family: Pentium Pro
 
Our solr data size can be increase in number like 90 laks. and writes per 
 day 
 will be around 1laks.   - Hope its possible by solr.
 
For write commit i have configured like
   
   1
   10
   
 
Is all above can be possible? 90laks datas and 1laks per day writes and 
 30laks per day read??  - if yes what type of system configuration would 
 require.
 
Please suggest us.
 
 thanks,
 Kalidoss.m,
   
 
 Get your world in your inbox!
 
 Mail, widgets, documents, spreadsheets, organizer and much more with your 
 Sifymail WIYI id!
 Log on to http://www.sify.com
 
 ** DISCLAIMER **
 Information contained and transmitted by this E-MAIL is proprietary to Sify 
 Limited and is intended for use only by the individual or entity to which it 
 is 
 addressed, and may contain information that is privileged, confidential or 
 exempt from disclosure under applicable law. If this is a forwarded message, 
 the 
 content of this E-MAIL may not have been sent with the authority of the 
 Company. 
 If you are not the intended recipient, an agent of the intended recipient or 
 a  
 person responsible for delivering the information to the named recipient,  
 you 
 are notified that any use, distribution, transmission, printing, copying or 
 dissemination of this information in any way or in any manner is strictly 
 prohibited. If you have received this communication in error, please delete 
 this 
 mail  notify us immediately at ad...@sifycorp.com



Re: Solr - Load Increasing.

2009-11-16 Thread Walter Underwood
Probably lakh: 100,000.

So, 900k qpd and 3M docs.

http://en.wikipedia.org/wiki/Lakh

wunder

On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:

 Hi,
 
 Your autoCommit settings are very aggressive.  I'm guessing that's what's 
 causing the CPU load.
 
 btw. what is laks?
 
 Otis
 --
 Sematext is hiring -- http://sematext.com/about/jobs.html?mls
 Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
 
 
 
 - Original Message 
 From: kalidoss kalidoss.muthuramalin...@sifycorp.com
 To: solr-user@lucene.apache.org
 Sent: Mon, November 16, 2009 9:11:21 AM
 Subject: Solr - Load Increasing. 
 
 Hi All.
 
   My server solr box cpu utilization  increasing b/w 60 to 90% and some time 
 solr is getting down and we are restarting it manually.
 
   No of documents in solr 30 laks.
   No of add/update requrest solr 30 thousand / day. Avg of every 30 minutes 
 around 500 writes.
   No of search request 9laks / day.
   Size of the data directory: 4gb.
 
 
   My system ram is 8gb.
   System available space 12gb.
   processor Family: Pentium Pro
 
   Our solr data size can be increase in number like 90 laks. and writes per 
 day 
 will be around 1laks.   - Hope its possible by solr.
 
   For write commit i have configured like
 
  1
  10
 
 
   Is all above can be possible? 90laks datas and 1laks per day writes and 
 30laks per day read??  - if yes what type of system configuration would 
 require.
 
   Please suggest us.
 
 thanks,
 Kalidoss.m,
 
 
 Get your world in your inbox!
 
 Mail, widgets, documents, spreadsheets, organizer and much more with your 
 Sifymail WIYI id!
 Log on to http://www.sify.com
 
 ** DISCLAIMER **
 Information contained and transmitted by this E-MAIL is proprietary to Sify 
 Limited and is intended for use only by the individual or entity to which it 
 is 
 addressed, and may contain information that is privileged, confidential or 
 exempt from disclosure under applicable law. If this is a forwarded message, 
 the 
 content of this E-MAIL may not have been sent with the authority of the 
 Company. 
 If you are not the intended recipient, an agent of the intended recipient or 
 a  
 person responsible for delivering the information to the named recipient,  
 you 
 are notified that any use, distribution, transmission, printing, copying or 
 dissemination of this information in any way or in any manner is strictly 
 prohibited. If you have received this communication in error, please delete 
 this 
 mail  notify us immediately at ad...@sifycorp.com
 



RE: Solr - Load Increasing.

2009-11-16 Thread Sudarsan, Sithu D.
 
Hi,

Lakh or Lac - 100,000
Crore   - 100,00,000 (ten million)

Commonly used in India

Sincerely,
Sithu D Sudarsan

-Original Message-
From: Walter Underwood [mailto:wun...@wunderwood.org] 
Sent: Monday, November 16, 2009 5:22 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr - Load Increasing.

Probably lakh: 100,000.

So, 900k qpd and 3M docs.

http://en.wikipedia.org/wiki/Lakh

wunder

On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:

 Hi,
 
 Your autoCommit settings are very aggressive.  I'm guessing that's
what's causing the CPU load.
 
 btw. what is laks?
 
 Otis
 --
 Sematext is hiring -- http://sematext.com/about/jobs.html?mls
 Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
 
 
 
 - Original Message 
 From: kalidoss kalidoss.muthuramalin...@sifycorp.com
 To: solr-user@lucene.apache.org
 Sent: Mon, November 16, 2009 9:11:21 AM
 Subject: Solr - Load Increasing. 
 
 Hi All.
 
   My server solr box cpu utilization  increasing b/w 60 to 90% and
some time 
 solr is getting down and we are restarting it manually.
 
   No of documents in solr 30 laks.
   No of add/update requrest solr 30 thousand / day. Avg of every 30
minutes 
 around 500 writes.
   No of search request 9laks / day.
   Size of the data directory: 4gb.
 
 
   My system ram is 8gb.
   System available space 12gb.
   processor Family: Pentium Pro
 
   Our solr data size can be increase in number like 90 laks. and
writes per day 
 will be around 1laks.   - Hope its possible by solr.
 
   For write commit i have configured like
 
  1
  10
 
 
   Is all above can be possible? 90laks datas and 1laks per day writes
and 
 30laks per day read??  - if yes what type of system configuration
would require.
 
   Please suggest us.
 
 thanks,
 Kalidoss.m,
 
 
 Get your world in your inbox!
 
 Mail, widgets, documents, spreadsheets, organizer and much more with
your 
 Sifymail WIYI id!
 Log on to http://www.sify.com
 
 ** DISCLAIMER **
 Information contained and transmitted by this E-MAIL is proprietary
to Sify 
 Limited and is intended for use only by the individual or entity to
which it is 
 addressed, and may contain information that is privileged,
confidential or 
 exempt from disclosure under applicable law. If this is a forwarded
message, the 
 content of this E-MAIL may not have been sent with the authority of
the Company. 
 If you are not the intended recipient, an agent of the intended
recipient or a  
 person responsible for delivering the information to the named
recipient,  you 
 are notified that any use, distribution, transmission, printing,
copying or 
 dissemination of this information in any way or in any manner is
strictly 
 prohibited. If you have received this communication in error, please
delete this 
 mail  notify us immediately at ad...@sifycorp.com
 



Re: Solr - Load Increasing.

2009-11-16 Thread Israel Ekpo
On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood wun...@wunderwood.orgwrote:

 Probably lakh: 100,000.

 So, 900k qpd and 3M docs.

 http://en.wikipedia.org/wiki/Lakh

 wunder

 On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:

  Hi,
 
  Your autoCommit settings are very aggressive.  I'm guessing that's what's
 causing the CPU load.
 
  btw. what is laks?
 
  Otis
  --
  Sematext is hiring -- http://sematext.com/about/jobs.html?mls
  Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
 
 
 
  - Original Message 
  From: kalidoss kalidoss.muthuramalin...@sifycorp.com
  To: solr-user@lucene.apache.org
  Sent: Mon, November 16, 2009 9:11:21 AM
  Subject: Solr - Load Increasing.
 
  Hi All.
 
My server solr box cpu utilization  increasing b/w 60 to 90% and some
 time
  solr is getting down and we are restarting it manually.
 
No of documents in solr 30 laks.
No of add/update requrest solr 30 thousand / day. Avg of every 30
 minutes
  around 500 writes.
No of search request 9laks / day.
Size of the data directory: 4gb.
 
 
My system ram is 8gb.
System available space 12gb.
processor Family: Pentium Pro
 
Our solr data size can be increase in number like 90 laks. and writes
 per day
  will be around 1laks.   - Hope its possible by solr.
 
For write commit i have configured like
 
   1
   10
 
 
Is all above can be possible? 90laks datas and 1laks per day writes
 and
  30laks per day read??  - if yes what type of system configuration would
 require.
 
Please suggest us.
 
  thanks,
  Kalidoss.m,
 
 
  Get your world in your inbox!
 
  Mail, widgets, documents, spreadsheets, organizer and much more with
 your
  Sifymail WIYI id!
  Log on to http://www.sify.com
 
  ** DISCLAIMER **
  Information contained and transmitted by this E-MAIL is proprietary to
 Sify
  Limited and is intended for use only by the individual or entity to
 which it is
  addressed, and may contain information that is privileged, confidential
 or
  exempt from disclosure under applicable law. If this is a forwarded
 message, the
  content of this E-MAIL may not have been sent with the authority of the
 Company.
  If you are not the intended recipient, an agent of the intended
 recipient or a
  person responsible for delivering the information to the named
 recipient,  you
  are notified that any use, distribution, transmission, printing, copying
 or
  dissemination of this information in any way or in any manner is
 strictly
  prohibited. If you have received this communication in error, please
 delete this
  mail  notify us immediately at ad...@sifycorp.com
 




Thanks Walter for clarifying that.

I too was wondering what laks meant.

It was a bit distracting when I read the original post.
-- 
Good Enough is not good enough.
To give anything less than your best is to sacrifice the gift.
Quality First. Measure Twice. Cut Once.


Re: Solr - Load Increasing.

2009-11-16 Thread Shashi Kant
I think it would be useful for members of this list to realize that not
everyone uses the same metrology and terms.

It is very easy for Americans to use the imperial system and presume
everyone does the same; Europeans to use the metric system etc. Hopefully
members on this list would be persuaded to use or at least clarify their
terminology.

While the apocryphal saying goes  the great thing about standards is they
are so many choose from, we should all make an effort to communicate across
cultures and nations.



On Mon, Nov 16, 2009 at 5:33 PM, Israel Ekpo israele...@gmail.com wrote:

 On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood wun...@wunderwood.org
 wrote:

  Probably lakh: 100,000.
 
  So, 900k qpd and 3M docs.
 
  http://en.wikipedia.org/wiki/Lakh
 
  wunder
 
  On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:
 
   Hi,
  
   Your autoCommit settings are very aggressive.  I'm guessing that's
 what's
  causing the CPU load.
  
   btw. what is laks?
  
   Otis
   --
   Sematext is hiring -- http://sematext.com/about/jobs.html?mls
   Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
  
  
  
   - Original Message 
   From: kalidoss kalidoss.muthuramalin...@sifycorp.com
   To: solr-user@lucene.apache.org
   Sent: Mon, November 16, 2009 9:11:21 AM
   Subject: Solr - Load Increasing.
  
   Hi All.
  
 My server solr box cpu utilization  increasing b/w 60 to 90% and
 some
  time
   solr is getting down and we are restarting it manually.
  
 No of documents in solr 30 laks.
 No of add/update requrest solr 30 thousand / day. Avg of every 30
  minutes
   around 500 writes.
 No of search request 9laks / day.
 Size of the data directory: 4gb.
  
  
 My system ram is 8gb.
 System available space 12gb.
 processor Family: Pentium Pro
  
 Our solr data size can be increase in number like 90 laks. and
 writes
  per day
   will be around 1laks.   - Hope its possible by solr.
  
 For write commit i have configured like
  
1
10
  
  
 Is all above can be possible? 90laks datas and 1laks per day writes
  and
   30laks per day read??  - if yes what type of system configuration
 would
  require.
  
 Please suggest us.
  
   thanks,
   Kalidoss.m,
  
  
   Get your world in your inbox!
  
   Mail, widgets, documents, spreadsheets, organizer and much more with
  your
   Sifymail WIYI id!
   Log on to http://www.sify.com
  
   ** DISCLAIMER **
   Information contained and transmitted by this E-MAIL is proprietary to
  Sify
   Limited and is intended for use only by the individual or entity to
  which it is
   addressed, and may contain information that is privileged,
 confidential
  or
   exempt from disclosure under applicable law. If this is a forwarded
  message, the
   content of this E-MAIL may not have been sent with the authority of
 the
  Company.
   If you are not the intended recipient, an agent of the intended
  recipient or a
   person responsible for delivering the information to the named
  recipient,  you
   are notified that any use, distribution, transmission, printing,
 copying
  or
   dissemination of this information in any way or in any manner is
  strictly
   prohibited. If you have received this communication in error, please
  delete this
   mail  notify us immediately at ad...@sifycorp.com
  
 
 


 Thanks Walter for clarifying that.

 I too was wondering what laks meant.

 It was a bit distracting when I read the original post.
 --
 Good Enough is not good enough.
 To give anything less than your best is to sacrifice the gift.
 Quality First. Measure Twice. Cut Once.



Re: Solr - Load Increasing.

2009-11-16 Thread Tom Alt
Nice to learn a new word for the day!

But to answer your question, or at least part of it, I don't really think
you want a configuration like

  autoCommit
  maxDocs1/maxDocs
  maxTime10/maxTime
  /autoCommit

Committing every doc, and every 10 milliseconds? That's just asking for
problems. How about starting with 1000 docs, and five minutes for maxTime
(5*60*1000) or about 3 laks of milliseconds.

That should help performance a lot. Try that, and see how it works.

Tom

On Mon, Nov 16, 2009 at 2:43 PM, Shashi Kant sk...@sloan.mit.edu wrote:

 I think it would be useful for members of this list to realize that not
 everyone uses the same metrology and terms.

 It is very easy for Americans to use the imperial system and presume
 everyone does the same; Europeans to use the metric system etc. Hopefully
 members on this list would be persuaded to use or at least clarify their
 terminology.

 While the apocryphal saying goes  the great thing about standards is they
 are so many choose from, we should all make an effort to communicate
 across
 cultures and nations.



 On Mon, Nov 16, 2009 at 5:33 PM, Israel Ekpo israele...@gmail.com wrote:

  On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood wun...@wunderwood.org
  wrote:
 
   Probably lakh: 100,000.
  
   So, 900k qpd and 3M docs.
  
   http://en.wikipedia.org/wiki/Lakh
  
   wunder
  
   On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:
  
Hi,
   
Your autoCommit settings are very aggressive.  I'm guessing that's
  what's
   causing the CPU load.
   
btw. what is laks?
   
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
   
   
   
- Original Message 
From: kalidoss kalidoss.muthuramalin...@sifycorp.com
To: solr-user@lucene.apache.org
Sent: Mon, November 16, 2009 9:11:21 AM
Subject: Solr - Load Increasing.
   
Hi All.
   
  My server solr box cpu utilization  increasing b/w 60 to 90% and
  some
   time
solr is getting down and we are restarting it manually.
   
  No of documents in solr 30 laks.
  No of add/update requrest solr 30 thousand / day. Avg of every 30
   minutes
around 500 writes.
  No of search request 9laks / day.
  Size of the data directory: 4gb.
   
   
  My system ram is 8gb.
  System available space 12gb.
  processor Family: Pentium Pro
   
  Our solr data size can be increase in number like 90 laks. and
  writes
   per day
will be around 1laks.   - Hope its possible by solr.
   
  For write commit i have configured like
   
 1
 10
   
   
  Is all above can be possible? 90laks datas and 1laks per day
 writes
   and
30laks per day read??  - if yes what type of system configuration
  would
   require.
   
  Please suggest us.
   
thanks,
Kalidoss.m,
   
   
Get your world in your inbox!
   
Mail, widgets, documents, spreadsheets, organizer and much more with
   your
Sifymail WIYI id!
Log on to http://www.sify.com
   
** DISCLAIMER **
Information contained and transmitted by this E-MAIL is proprietary
 to
   Sify
Limited and is intended for use only by the individual or entity to
   which it is
addressed, and may contain information that is privileged,
  confidential
   or
exempt from disclosure under applicable law. If this is a forwarded
   message, the
content of this E-MAIL may not have been sent with the authority of
  the
   Company.
If you are not the intended recipient, an agent of the intended
   recipient or a
person responsible for delivering the information to the named
   recipient,  you
are notified that any use, distribution, transmission, printing,
  copying
   or
dissemination of this information in any way or in any manner is
   strictly
prohibited. If you have received this communication in error, please
   delete this
mail  notify us immediately at ad...@sifycorp.com
   
  
  
 
 
  Thanks Walter for clarifying that.
 
  I too was wondering what laks meant.
 
  It was a bit distracting when I read the original post.
  --
  Good Enough is not good enough.
  To give anything less than your best is to sacrifice the gift.
  Quality First. Measure Twice. Cut Once.