Maximum solr processes per machine

2013-09-29 Thread adfel70
Hi, I'm thinking of solr cluster architecture before purchasing machines. My total index size is around 5TB. I want to have replication factor of 3. total 15TB. I've understood that I should have 50-100% of the index size as ram, for OS cache. Lets say we're talking about around 10TB of memory.

Re: Maximum solr processes per machine

2013-09-29 Thread Erick Erickson
bq: is there an upper limit of amount of solr processes per machine, No, assuming they're all in separate JVMs. I've see reports, though, that increasing the number of JVMs past the number of CPU cores gets into "iffy" territory. And, depending on your disk storage they may all be contending for

Re: Maximum solr processes per machine

2013-09-29 Thread adfel70
How can I configure the disk storage so that disk access is optimized? I'm considering having RAID-10 and I think I'll have arround 4-8 disks per machine. Should I run each solr jvm to point on a datadir on differnet disks, or is there some other way to optimize this? Erick Erickson wrote > bq:

ClusteringComponent under Tomcat 7

2013-09-29 Thread Lieberman, Ariel
Hi, I'm trying to run Solr 4.3 (and 4.4) with -Dsolr.clustering.enabled=true I've copied all relevant jars to ./lib directory under the instance. With jetty it runs OK! But, under Tomcat I receives the error (exception) below. Any idea/help? Thanks, -Ariel org.apache.solr.common.SolrExcepti

Re: Maximum solr processes per machine

2013-09-29 Thread Bram Van Dam
On 09/29/2013 04:03 PM, adfel70 wrote: How can I configure the disk storage so that disk access is optimized? I'm considering having RAID-10 and I think I'll have arround 4-8 disks per machine. Should I run each solr jvm to point on a datadir on differnet disks, or is there some other way to opti

Re: XPathEntityProcessor nested in TikaEntityProcessor query null exception

2013-09-29 Thread Andreas Owen
how dum can you get. obviously quite dum... i would have to analyze the html-pages with a nested instance like this: but i'm pretty sure the foreach i

Re: Hello and help :)

2013-09-29 Thread Matheus Salvia
Thanks for the anwser. Yes, you understood it correctly. The method you proposed should work perfectly, except I do have one more requirement that I forgot to mention earlier, and I apologize for that. The true problem we are facing is: * find all documents for userID=x, where userID=x has more tha

Nagle's Algorithm

2013-09-29 Thread William Bell
How do I set TCP_NODELAY on the http sockets for Jetty in SOLR 4? Is there an option in jetty.xml ? /* Create new stream socket */ sock = *socket*( AF_INET, SOCK_STREAM, 0 ); /* Disable the Nagle (TCP No Delay) algorithm */ flag = 1; ret = *setsockopt*( sock, IPPROTO_TCP, TCP_NODELAY, (char

Re: Nagle's Algorithm

2013-09-29 Thread Dan Davis
I don't keep up with this list well enough to know whether anyone else answered. I don't know how to do it in jetty.xml, but you can certainly tweak the code. java.net.Socket has a method setTcpNoDelay() that corresponds with the standard Unix system calls. Long-time past, my suggestion of this

Re: Nagle's Algorithm

2013-09-29 Thread Michael Sokolov
I dunno, but this makes it look as if this may already be taken care of: http://jira.codehaus.org/browse/JETTY-1196 On 9/29/2013 9:22 PM, William Bell wrote: How do I set TCP_NODELAY on the http sockets for Jetty in SOLR 4? Is there an option in jetty.xml ? /* Create new stream socket */ soc

Re: Maximum solr processes per machine

2013-09-29 Thread Shawn Heisey
On 9/29/2013 7:21 AM, adfel70 wrote: > Hi, > I'm thinking of solr cluster architecture before purchasing machines. > > > My total index size is around 5TB. I want to have replication factor of 3. > total 15TB. > I've understood that I should have 50-100% of the index size as ram, for OS > cache.

Re: Nagle's Algorithm

2013-09-29 Thread Shawn Heisey
On 9/29/2013 7:22 PM, William Bell wrote: > How do I set TCP_NODELAY on the http sockets for Jetty in SOLR 4? The client usually makes that decision, not the server. This parameter is turned on by default for recent HttpClient versions, the library used by SolrJ. Even the JETTY issue uncovered b

Re: Update field properties via Schema Rest API ?

2013-09-29 Thread Upayavira
Updating a field isn't straightforward. Changing size from string to int would, if you don't re-index, break your index. The schema tells Slr how to interpret the binary bits it finds in the index. If there are no bits in the index for that field name, then no issue. If there already are bits in th

Re: Hello and help :)

2013-09-29 Thread Upayavira
If your app and solr aren't far apart, you shouldn't be afraid of multiple queries to solr per user request (I once discovered an app that did 36 hits to solr per user request, and despite such awfulness of design, no user ever complained about speed). You could do a query to solr for q=+user_id:X

Re: Maximum solr processes per machine

2013-09-29 Thread adfel70
Bram Van Dam wrote > On 09/29/2013 04:03 PM, adfel70 wrote: > If you're doing real time on a 5TB index then you'll probably want to > throw your money at the fastest storage you can afford (SSDs vs spinning > rust made a huge difference in our benchmarks) and the fastest CPUs you > can get your