Re: Max indexing threads & RamBuffered size

2015-12-05 Thread Erick Erickson
bq: What adds bottleneck in the indexing flow? Is it the buffering and flushing out to disk ? It Depends (tm). What do the Solr logs show when one of these two things happens? You pretty much have to put a profiler on the Solr instance to see where it's spending the time, but timeouts are very

Re: Solrcloud: 1 server, 1 configset, multiple collections, multiple schemas

2015-12-05 Thread Erick Erickson
You have to upload the different configset with the zookeeper client (this is done for you when you do the examples) using zkcli, see the "upconfig" command here; https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities Similarly, you need to make changes locally (perhaps after

Re: solr with Isilon HDFS

2015-12-05 Thread Gaurav Patel
Got it. Thanks for the help Erick. On Sat, Dec 5, 2015 at 1:02 PM, Erick Erickson wrote: > bq: Should zookeeper be installed along with solr on each box or should be > installed in separate 2 Virtual machines by itself? > > Zookeeper should be installed on an

Re: solr with Isilon HDFS

2015-12-05 Thread Erick Erickson
bq: Should zookeeper be installed along with solr on each box or should be installed in separate 2 Virtual machines by itself? Zookeeper should be installed on an odd number of machines as it requires a quorum which is (Number of zookeeper nodes)/2 + 1. With two ZKs, if either of them fail

Re: Max indexing threads & RamBuffered size

2015-12-05 Thread Erick Erickson
I'm pretty sure that max indexing threads is per core, but just looked and it's not supported in Solr 5.3 and above so I wouldn't worry about it at all. I've never seen much in the way of benefit for bumping this past 128M or maybe 256M. This is just how much memory is filled up before the buffer

Re: schema fileds and Typefield in solr-5.3.1

2015-12-05 Thread kostali hassan
I fixe the prb using requestHandler dataimoprt: tika-data-config.xml I configure the tika-data-config.xml according to my needs to get the right value : now dont need indexing from Commandline using

Re: solr with Isilon HDFS

2015-12-05 Thread Gaurav Patel
Thanks Toke. Your input has been informative and valuable. I will go through the links you provided and will let you know what we end up going. On Sat, Dec 5, 2015 at 5:02 AM, Toke Eskildsen wrote: > Gaurav Patel wrote: > > 3 Physical Machines

Re: Sold 5.3 with Basic Authentication in a not SolrCloud environment

2015-12-05 Thread Ishan Chattopadhyaya
At the moment, the Basic Authentication doesn't work with standalone Solr (non-SolrCloud). Kerberos authentication works. If you want support for Basic Authentication for standalone Solr, please feel free to open a JIRA and contribute a patch. On Wed, Sep 16, 2015 at 3:30 PM, Dominique Bejean

Re: Authorization API versus zkcli.sh

2015-12-05 Thread Ishan Chattopadhyaya
As I see things, if you're using Basic Authentication and/or Rule Based Authorization, then the security.json could be copied over to start things. On Sat, Dec 5, 2015 at 3:38 AM, Oakley, Craig (NIH/NLM/NCBI) [C] < craig.oak...@nih.gov> wrote: > Looking through >

Re: How to config security.json?

2015-12-05 Thread Ishan Chattopadhyaya
FYI, http://lucidworks.com/blog/2015/08/17/securing-solr-basic-auth-permission-rules/ On Thu, Nov 19, 2015 at 4:21 PM, Byzen Ma wrote: > Hi, I'm not quite familar with security.json. I want to achieve these > implementations. (1)Anyone who wants to do read/select/query

Re: solr with Isilon HDFS

2015-12-05 Thread Toke Eskildsen
Gaurav Patel wrote: > 3 Physical Machines with 60 cpu cores and 512 GB RAM each. > EMC Isilon Appliance with PB storage. It can be accessed via HDFS or NFS. We have experimented a little bit with smaller machines, backed by EMC Isilon over NFS. That worked surprisingly well,

Re: Solrcloud: 1 server, 1 configset, multiple collections, multiple schemas

2015-12-05 Thread bengates
I understand. How to do this via the API? -- View this message in context: http://lucene.472066.n3.nabble.com/Solrcloud-1-server-1-configset-multiple-collections-multiple-schemas-tp4243584p4243737.html Sent from the Solr - User mailing list archive at Nabble.com.

how to control location of solr PID file

2015-12-05 Thread tedsolr
I'm running v5.2.1 on red hat linux. The "solr status" command is not recognizing all my nodes. Consequently, "solr stop -all" only stops the node on 8983. What's the recommended file structure for placing multiple nodes on the same host? I am trying multiple "solr" folders within the same solr

Re: How to config security.json?

2015-12-05 Thread Anshum Gupta
Hi Byzen, To begin with, seems like you didn't add a user for the groups/roles you have defined above in your security.json. You would need to do so in order to be able to send acceptable read/update requests. In addition, there were a couple of known issues that were reported and fixed and

Re: Max indexing threads & RamBuffered size

2015-12-05 Thread KNitin
I have an extremely large indexing load (per doc size of 4-5 Mb with over 100M docs). I have auto commit settings to flush to disk (with open searcher as false) every 20 seconds. Even with that the update sometime fails or timed out. The goal is to improve the indexing throughput and hence trying