restore failure on Solr 6.0.0

2016-08-24 Thread Siddhartha Singh Sandhu
Hi, The API to backup(action=BACKUP) has not been implemented for Solr 6.0.0. I used the replication end pt. to create a backup which it did. When I try to restore from backup Solr throughs: "exception":"java.lang.OutOfMemoryError: Java heap space" I would believe that the amount of memory

Any Performance Study to show Effect Of Shards with Replica on QPS

2016-06-01 Thread Siddhartha Singh Sandhu
Hi, I wanted to know anyone had a link to graphical study showing the correlation between Shards and Replicas against QPS. I have this link but this show this link: http://www.slideshare.net/thelabdude/solr-performance which compare indexing performance between Shards and replicas. I know that

Re: SolrCloud Shard console shows roughly same number of documents?

2016-05-31 Thread Siddhartha Singh Sandhu
Hi, I was speculating whether sharding is done on: 1. index terms with each shard having the whole document space. 2. document space with each shard have num(documents/no. of shards) of the documents divided between them. Regards, Sid. On Tue, May 31, 2016 at 9:27 AM, Siddhartha Singh Sandhu

Re: Clarity on Sharding Concepts.

2016-05-31 Thread Siddhartha Singh Sandhu
Hi Mugeesh, I was speculating whether sharding is done on: 1. index terms with each shard having the whole document space. 2. document space with each shard have num(documents/no. of shards) of the documents divided between them. Regards, Sid. On Tue, May 31, 2016 at 12:19 AM, Mugeesh Husain

Re: Clarity on Sharding Concepts.

2016-05-31 Thread Siddhartha Singh Sandhu
Thank you Mugeesh. On Tue, May 31, 2016 at 12:19 AM, Mugeesh Husain wrote: > Hi, > > To read out this document > > https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud > for proper understanding. > > FYI, you are using implicit router, a

Clarity on Sharding Concepts.

2016-05-30 Thread Siddhartha Singh Sandhu
Hi Community, I need some help understanding some concepts. I have the config on 2 severs: 2 shards each with 1 replica. Hence, on each server I have 1. shard1_replica1 2 . shard2_replica1 Suppose I have 50 documents then, shard1_replica1 + shard2_replica1 = 50 ? or shard2_replica1 = 50 &&

Re: SolrCloud Shard console shows roughly same number of documents?

2016-05-28 Thread Siddhartha Singh Sandhu
Still struggling with this. Bump. :) On Thu, May 26, 2016 at 3:53 PM, Siddhartha Singh Sandhu < sandhus...@gmail.com> wrote: > Hi Erick, > > Thank you for the reply. What I meant was suppose I have the config: > > 2 shards each with 1 replica. > > Hence,

Re: SolrCloud Shard console shows roughly same number of documents?

2016-05-26 Thread Siddhartha Singh Sandhu
nce rather than one-at-a-time. > > Best, > Erick > > On Wed, May 25, 2016 at 1:54 PM, Siddhartha Singh Sandhu > <sandhus...@gmail.com> wrote: > > Hi, > > > > I recently moved to a SolrCloud config. I had a few questions: > > > > Q1. Does a shard

Re: "data import handler : import data from sql database :how to search in all fields"

2016-05-26 Thread Siddhartha Singh Sandhu
The schema.xml/managed_schema defines the default search field as `text`. You can make all fields that you want searchable type `text`. On Thu, May 26, 2016 at 10:23 AM, kostali hassan wrote: > I import data from sql databases with DIH . I am looking for serch term

Re: how can we use multi term search along with stop words

2016-05-26 Thread Siddhartha Singh Sandhu
Hi Preeti, You can use the analysis tool in the Solr console to see how your queries are being tokenized. Based on your results you might need to make changes in "strings_ci". Also, If you want to be able to search on stopwords you might want to remove solr.StopFilterFactory from indexing and

SolrCloud Shard console shows roughly same number of documents?

2016-05-25 Thread Siddhartha Singh Sandhu
Hi, I recently moved to a SolrCloud config. I had a few questions: Q1. Does a shard show cumulative number of documents or documents present in that particular shard on the admin console of respective shard? Q2. If 1's answer is non-cumulative then my shards(on different servers) are indexing

Re: Import html data in mysql and map schemas using only Solr CELL+TIKA+DIH [scottchu]

2016-05-20 Thread Siddhartha Singh Sandhu
You will have to configure your schema.xml in Solr. What version are you using? On Fri, May 20, 2016 at 2:17 AM, scott.chu wrote: > > I have a mysql table with over 300M blog articles. The records are in html > format. Is it possible to import these records using only

Re: load schema.xml

2016-05-18 Thread Siddhartha Singh Sandhu
Hi, Add the following to your solrconfig.xml: in place of: because solr 6.0.0 defaults to managed schema. First time answering and not questioning. Community, please correct me if I am wrong. Sid. On Wed, May 18, 2016 at 8:48 AM, forme book wrote: > Hi, > >

Re: id field always stored?

2016-05-06 Thread Siddhartha Singh Sandhu
notes (under 5.5 I believe) for > more details. > > Regards, >Alex. > > Newsletter and resources for Solr beginners and intermediates: > http://www.solr-start.com/ > > > On 6 May 2016 at 06:30, Siddhartha Singh Sandhu <sandhus...@gmail.com> > wrote: &

id field always stored?

2016-05-05 Thread Siddhartha Singh Sandhu
Hi, 1. I was doing some exploration and wanted to know if the id field is always stored even when I set stored = false. ** 2. Also, even though I removed dynamic fields, anything tagged *_id is getting stored despite marking that field stored = false. ** Where string is defined as:

Re: Pressed optimize and now SOLR is not indexing while optimize is going on

2015-10-07 Thread Siddhartha Singh Sandhu
Thanks from my end too. And thanks for the question Eric that added a lot to my understanding as well. Regards. Sid. On Wed, Oct 7, 2015 at 10:04 AM, Eric Torti wrote: > Cool, Toke and Shawn! > > That's exactly what I was looking for. I'll have a look at those resources >

Re: Pressed optimize and now SOLR is not indexing while optimize is going on

2015-10-06 Thread Siddhartha Singh Sandhu
the optimize button? Thanks Sid. On Tue, Oct 6, 2015 at 6:55 AM, Toke Eskildsen <t...@statsbiblioteket.dk> wrote: > On Mon, 2015-10-05 at 17:26 -0400, Siddhartha Singh Sandhu wrote: > > Following up on that: Would having an SSD make considerable difference in > > speed? > &g

Re: Pressed optimize and now SOLR is not indexing while optimize is going on

2015-10-06 Thread Siddhartha Singh Sandhu
eader":{"status":0,"QTime":57}} Thank you. Sid. On Tue, Oct 6, 2015 at 3:21 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 10/6/2015 8:18 AM, Siddhartha Singh Sandhu wrote: > > A have a few questions about optimize. Is the search index fully > searchable

Re: How to disable the admin interface

2015-10-05 Thread Siddhartha Singh Sandhu
Help please? On Sun, Oct 4, 2015 at 5:07 PM, Siddhartha Singh Sandhu < sandhus...@gmail.com> wrote: > Hi Shawn and Andrew, > > I am on page with you guys about the ssh authentication and communicating > with the API's that SOLR has to provide. I simply don't want the GUI

Re: How to disable the admin interface

2015-10-05 Thread Siddhartha Singh Sandhu
able to > > diagnose the problem. > > > > wunder > > Walter Underwood > > wun...@wunderwood.org > > http://observer.wunderwood.org/ (my blog) > > > > > On Oct 5, 2015, at 11:34 AM, Siddhartha Singh Sandhu < > sandhus...@gmail.com> wrote: > > >

Pressed optimize and now SOLR is not indexing while optimize is going on

2015-10-05 Thread Siddhartha Singh Sandhu
Hi, I pressed the optimize switch. Wasn't the best decision I made today. The aftermath of it was that when I tried to index more documents the curl just waited and waited. I pinged my SOLR and all is well. I am able to access the admin console also. I can query the SOLR machine too. But, I

Re: Pressed optimize and now SOLR is not indexing while optimize is going on

2015-10-05 Thread Siddhartha Singh Sandhu
at your disk have free _at least_ as > much space as the index takes up. > > It may just be that the disk is so busy with the optimize (it's mostly > just writing from one file to another) that it's appearing to be > slow... > > Best, > Erick > > On Mon, Oct 5, 2015 at 1

Re: Pressed optimize and now SOLR is not indexing while optimize is going on

2015-10-05 Thread Siddhartha Singh Sandhu
Thank you Eric. I think your explanation was the reason. Following up on that: Would having an SSD make considerable difference in speed? On Mon, Oct 5, 2015 at 5:18 PM, Siddhartha Singh Sandhu < sandhus...@gmail.com> wrote: > Scrap the last one. It just took 10 mins to load. I pan

Re: Pressed optimize and now SOLR is not indexing while optimize is going on

2015-10-05 Thread Siddhartha Singh Sandhu
Scrap the last one. It just took 10 mins to load. I panicked too quick. On Mon, Oct 5, 2015 at 5:15 PM, Siddhartha Singh Sandhu < sandhus...@gmail.com> wrote: > I restarted my SOLR and it is now not reloading the configuration. > Is my solr index corrupted? > > Sid. > >

Re: How to disable the admin interface

2015-10-04 Thread Siddhartha Singh Sandhu
olr or reach the admin. > > > King Regards, > Andrea Roggerone > > > On 04/ott/2015, at 08:11, Shawn Heisey <apa...@elyograg.org> wrote: > > > >> On 10/3/2015 9:17 PM, Siddhartha Singh Sandhu wrote: > >> I want to disable the admin interface in SOL

RE: How to disable the admin interface

2015-10-03 Thread Siddhartha Singh Sandhu
Hi, I want to disable the admin interface in SOLR. I understand that authentication is available in the solrcloud mode but until that happens I want to disable the admin interface in my prod environment. How can I do this? Regards. Sid.

Using a plugin to filter in schema.xml

2015-09-25 Thread Siddhartha Singh Sandhu
Hi, I wanted to use the twitter-text libraries github implementation to filter the tokens(hashtags) in my text. I know I can use the Pattern Matching tokenizer also, but would trust twitter's library more then my own regex to do the job for me. I wanted to use it in unison with the

Re: Using a plugin to filter in schema.xml

2015-09-25 Thread Siddhartha Singh Sandhu
I need a go to for writing the custom tokenizer. any suggestions? On Fri, Sep 25, 2015 at 2:36 PM, Siddhartha Singh Sandhu < sandhus...@gmail.com> wrote: > For sure. > > On Fri, Sep 25, 2015 at 1:13 PM, Alexandre Rafalovitch <arafa...@gmail.com > > wrote: > >>

Re: Using a plugin to filter in schema.xml

2015-09-25 Thread Siddhartha Singh Sandhu
to open-source if you managed > to achieve that. > > Regards, >Alex. > > Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: > http://www.solr-start.com/ > > > On 25 September 2015 at 12:58, Siddhartha Singh Sandhu > <sandhus...@gmail.com&

Different ports for search and upload request

2015-09-24 Thread Siddhartha Singh Sandhu
Hi, I wanted to know if we can configure different ports as end points for uploading and searching API. Also, if someone could point me in the right direction. Regards, Sid.

Re: Different ports for search and upload request

2015-09-24 Thread Siddhartha Singh Sandhu
resources among-st solr threads. Regards, Sid. On Thu, Sep 24, 2015 at 5:17 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 9/24/2015 2:01 PM, Siddhartha Singh Sandhu wrote: > > I wanted to know if we can configure different ports as end points for > > uploading an

Re: Different ports for search and upload request

2015-09-24 Thread Siddhartha Singh Sandhu
o user directly but > via application / service / api. > > Thanks, > Susheel > > On Thu, Sep 24, 2015 at 4:01 PM, Siddhartha Singh Sandhu < > sandhus...@gmail.com> wrote: > > > Hi, > > > > I wanted to know if we can configure different ports as end points fo

Re: Mapping external JSON fields in DIHconfig.xml

2015-09-23 Thread Siddhartha Singh Sandhu
aller chunks and/or > using SolrJ to parse the file with a sax-like parser and index docs > in smaller chunks (I often use 1,000 docs at a time, but it depends on > how big they are). > > Best, > Erick > > On Tue, Sep 22, 2015 at 12:43 PM, Siddhartha Singh Sandhu > <sandhus..

RE: Mapping external JSON fields in DIHconfig.xml

2015-09-22 Thread Siddhartha Singh Sandhu
Hi, I am relatively new to Solr and have usage query. I have a 20 GB JSON file which I want to upload into my solr. Do I have to form smaller chunks or is there a way to upload the whole thing in one go? I am getting the following error with bin/post: Entering auto mode. File endings considered