RE: Issue using filter query with spellCheck component

2010-12-31 Thread taimurAQ

Thanks for the reply.

What i have done now is that i take the suggested string and make another
query to solr along with the filter parameter.

It is working for now, since i can't figure out another workaround.

Regards,
Taimur
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-using-filter-query-with-spellCheck-component-tp2166322p2172428.html
Sent from the Solr - User mailing list archive at Nabble.com.


Spelling Suggestions vs Correction

2010-12-31 Thread TxCSguy

Hi,

I am trying to clear up some confusion about SOLR's spell check
functionality.  Being new to SORL and Lucene as well, I was under the
assumption that spellcheck would take a query entered by a user and end up
actually querying the index based upon the corrections returned by the
spellcheck component.  

For example (referring to the sample data that comes with SOLR),  let's say
that the user issues a search for sell when what they really are looking
for is dell.  So the spellcheck component is returning the suggestion of
dell (I can see this in the response) but still searching the index for
sell.

So my questions are:
1) Is my example above a correct representation of what is happening by
incorporating the spellcheck component in?
2) Is there a way to actually search for the corrected version (dell) rather
than the original version (sell)?

Thanks in advance for any help,
Mark 
  


-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Spelling-Suggestions-vs-Correction-tp2172572p2172572.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Different behavior for q=goo.com vs q=@goo.com in queries?

2010-12-31 Thread Gora Mohanty
On Fri, Dec 31, 2010 at 2:40 AM, mrw mikerobertsw...@gmail.com wrote:


 Basically, just what you've suggested.  I did the field/query analysis piece
 with verbose output.  Not entirely sure how to interpret the results, of
 course.  Currently reading anything I can find on that.
[...]

From the above, it is not quite clear whether you followed the
import of Erick's reply.

Essentially, at this point, we are guessing in the dark, but Erick
is very likely correct that this is an (unexpected) consequence of
your analysis chain at index/query time. The admin/analysis page
ought to show you how goo.com is indexed/queried vs. how
@goo.com is indexed/queried.

Alternatively, please show us  the index/query analysis chain is
configured in your schema.xml.

Regards,
Gora


How to delete links from Index

2010-12-31 Thread Tapan . Sadafal
Dear,

I have created Index through Crawler Solr but i am getting old pages link 
also. 
My query is how to delete specific links from Index through Solr Admin?

Regards,
Tapan Sadafal.
DID : 67897880
This e-mail is confidential. It may also be legally privileged. If you are 
not the addressee you may not copy, forward, disclose or use any part of 
it. If you have received this message in error, please delete it and all 
copies from your system and notify the sender immediately by return 
e-mail. Internet communications cannot be guaranteed to be timely secure, 
error or virus-free. The sender does not accept liability for any errors 
or omissions. 

Re: Custom Analyzer

2010-12-31 Thread nitishgarg

My actual class files present in the jar file are:
MarathiAnalyzer.class
MarathiStemFilter.class
MarathiStemmer.class
MarathiAnayzer$1.class
MarathiAnalyzer$SavedStreams.class

Please tell what else do I need to specify about my problem?
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Custom-Analyzer-tp2162710p2167456.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Spelling Suggestions vs Correction

2010-12-31 Thread Erick Erickson
Here's a discussion of the difference between them, does that answer?

http://lucene.472066.n3.nabble.com/spell-check-vs-terms-component-td1870214.html

Best
Erick

On Fri, Dec 31, 2010 at 8:55 AM, TxCSguy markefonte...@gmail.com wrote:


 Hi,

 I am trying to clear up some confusion about SOLR's spell check
 functionality.  Being new to SORL and Lucene as well, I was under the
 assumption that spellcheck would take a query entered by a user and end up
 actually querying the index based upon the corrections returned by the
 spellcheck component.

 For example (referring to the sample data that comes with SOLR),  let's say
 that the user issues a search for sell when what they really are looking
 for is dell.  So the spellcheck component is returning the suggestion of
 dell (I can see this in the response) but still searching the index for
 sell.

 So my questions are:
 1) Is my example above a correct representation of what is happening by
 incorporating the spellcheck component in?
 2) Is there a way to actually search for the corrected version (dell)
 rather
 than the original version (sell)?

 Thanks in advance for any help,
 Mark



 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Spelling-Suggestions-vs-Correction-tp2172572p2172572.html
 Sent from the Solr - User mailing list archive at Nabble.com.



Re: Custom Analyzer

2010-12-31 Thread Erick Erickson
Well, if that's what's in your class, this won't work:
analyzer
type=index class=org.apache.lucene.analysis.mr.MarathiAnalyzer

because it's looking for org.apache. You can try just
class=MarathiAnalyzer

So I'm not sure removing the package statement is really what you want here.

So now I'm wondering if you really put the jar file in the right place, is
it possible
that that's the problem?

Best
Erick

On Thu, Dec 30, 2010 at 8:04 AM, nitishgarg nitishgarg1...@gmail.comwrote:


 My actual class files present in the jar file are:
 MarathiAnalyzer.class
 MarathiStemFilter.class
 MarathiStemmer.class
 MarathiAnayzer$1.class
 MarathiAnalyzer$SavedStreams.class

 Please tell what else do I need to specify about my problem?
 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Custom-Analyzer-tp2162710p2167456.html
 Sent from the Solr - User mailing list archive at Nabble.com.



Re: Different behavior for q=goo.com vs q=@goo.com in queries?

2010-12-31 Thread Ryan McKinley
also try debugQuery=true and see why each result matched



On Thu, Dec 30, 2010 at 4:10 PM, mrw mikerobertsw...@gmail.com wrote:


 Basically, just what you've suggested.  I did the field/query analysis piece
 with verbose output.  Not entirely sure how to interpret the results, of
 course.  Currently reading anything I can find on that.


 Thanks


 Erick Erickson wrote:

 What steps have you taken to figure out whether the
 contents of your index are what you think? I suspect
 that the fields you're indexing aren't being
 analyzed/tokenized quite the way you expect either at
 query time or index time (or maybe both!).

 Take a look at the admin/analysis page for the field you're indexing
 the data into. If that doesn't shed any light on the problem,
 please paste in the fieldType definition for the field in question,
 maybe another set of eyes can see the issue.

 Best
 Erick





 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Different-behavior-for-q-goo-com-vs-q-goo-com-in-queries-tp2168935p2169478.html
 Sent from the Solr - User mailing list archive at Nabble.com.



Re: How to delete links from Index

2010-12-31 Thread Lance Norskog
The Solr admin pages do not have a delete function. You have to use
'curl' or 'wget' or your own SolrJ program to delete documents.

On Fri, Dec 31, 2010 at 3:34 AM,  tapan.sada...@hsbcinv.com wrote:
 Dear,

 I have created Index through Crawler Solr but i am getting old pages link
 also.
 My query is how to delete specific links from Index through Solr Admin?

 Regards,
 Tapan Sadafal.
 DID : 67897880
 This e-mail is confidential. It may also be legally privileged. If you are
 not the addressee you may not copy, forward, disclose or use any part of
 it. If you have received this message in error, please delete it and all
 copies from your system and notify the sender immediately by return
 e-mail. Internet communications cannot be guaranteed to be timely secure,
 error or virus-free. The sender does not accept liability for any errors
 or omissions.



-- 
Lance Norskog
goks...@gmail.com


solr newbie: Diagnose why DataImportHandler DIH not saving documents

2010-12-31 Thread Stephen Boesch
I am asking for a full DataImport via a url.  It seems to be partially
 happy with the request - with debug=on I can see it saying that 10
documents were indexed.  The backend however realizes there are actually 440
records available for the query.

Not sure why only 10 records were selected and then why even those 10
records are not stored.


Here is the obfuscated url used for invoking the DataImport:

mySolrHost:8983/solr/core0/dataimport?command=full-importdebug=onhttp://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=on


Here is the output:  looks reasonable for the 10 records it does find:
notice it says *added/updated 10 documents*

0360db-data-config.xmlfull-importdebugBrad is testing
thisjava.math.BigDecimal:1java.math.BigDecimal:15000947 Wood Duck
Lanejava.math.BigDecimal:3java.math.BigDecimal:15002Stanford Quad
Sculpturejava.math.BigDecimal:3java.math.BigDecimal:15200Apple Store - Palo
Altojava.math.BigDecimal:3java.math.BigDecimal:15201Fox
Theaterjava.math.BigDecimal:3java.math.BigDecimal:15220java.math.BigDecimal:3java.math.BigDecimal:15222Knowtate
promojava.math.BigDecimal:4welcome to Knowtatejava.math.BigDecimal:16163The
Green Dragon Tavernjava.math.BigDecimal:5java.math.BigDecimal:15020The All
New Infiniti Mjava.math.BigDecimal:5Introjava.math.BigDecimal:15100The All
New Infiniti Mjava.math.BigDecimal:5To hear current
specialsjava.math.BigDecimal:15100idleConfiguration Re-loaded
sucessfully11002010-12-31 16:45:11Indexing completed. *Added/Updated: 10
documents. *Deleted 0 documents.100:0:0.331This response format is
experimental. It is likely to change in the future.


But when I go to the Admin screen, it tells me   Documents Processed:
10   *Total
Documents Processed 0*
*
*
So what is difference between Documents and Total Documents ??  Note that
there is presently *no *data in the indexes.

mySolrHost:8983/solr/core0/admin/stats.jsphttp://knowtate.servehttp.com:8983/solr/core0/admin/

*name: */dataimport  *class:
*org.apache.solr.handler.dataimport.DataImportHandler
 *version: *1.0  *description: *Manage data import from databases to Solr  *
stats: *Status : IDLE
Documents Processed : 10
Requests made to DataSource : 1
Rows Fetched : 10
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 0
Total Requests made to DataSource : 0
Total Rows Fetched : 0
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1293831460260
requests : 2


Re: solr newbie: Diagnose why DataImportHandler DIH not saving documents

2010-12-31 Thread Stephen Boesch
one little extra piece of info: part of the stats page got omitted - notably
the number of errors was reported as 0.

errors : 0
timeouts : 0
totalTime : 1963
avgTimePerRequest : 981.5
avgRequestsPerSecond : 0.0011371888


2010/12/31 Stephen Boesch java...@gmail.com

 I am asking for a full DataImport via a url.  It seems to be partially
  happy with the request - with debug=on I can see it saying that 10
 documents were indexed.  The backend however realizes there are actually 440
 records available for the query.

 Not sure why only 10 records were selected and then why even those 10
 records are not stored.


 Here is the obfuscated url used for invoking the DataImport:

 mySolrHost:8983/solr/core0/dataimport?command=full-importdebug=onhttp://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=on


 Here is the output:  looks reasonable for the 10 records it does find:
 notice it says *added/updated 10 documents*

 0360db-data-config.xmlfull-importdebugBrad is testing
 thisjava.math.BigDecimal:1java.math.BigDecimal:15000947 Wood Duck
 Lanejava.math.BigDecimal:3java.math.BigDecimal:15002Stanford Quad
 Sculpturejava.math.BigDecimal:3java.math.BigDecimal:15200Apple Store - Palo
 Altojava.math.BigDecimal:3java.math.BigDecimal:15201Fox
 Theaterjava.math.BigDecimal:3java.math.BigDecimal:15220java.math.BigDecimal:3java.math.BigDecimal:15222Knowtate
 promojava.math.BigDecimal:4welcome to Knowtatejava.math.BigDecimal:16163The
 Green Dragon Tavernjava.math.BigDecimal:5java.math.BigDecimal:15020The All
 New Infiniti Mjava.math.BigDecimal:5Introjava.math.BigDecimal:15100The All
 New Infiniti Mjava.math.BigDecimal:5To hear current
 specialsjava.math.BigDecimal:15100idleConfiguration Re-loaded
 sucessfully11002010-12-31 16:45:11Indexing completed. *Added/Updated: 10
 documents. *Deleted 0 documents.100:0:0.331This response format is
 experimental. It is likely to change in the future.


 But when I go to the Admin screen, it tells me   Documents Processed: 10
 *Total Documents Processed 0*
 *
 *
 So what is difference between Documents and Total Documents ??  Note that
 there is presently *no *data in the indexes.

 mySolrHost:8983/solr/core0/admin/stats.jsphttp://knowtate.servehttp.com:8983/solr/core0/admin/

 *name: * /dataimport   *class: * 
 org.apache.solr.handler.dataimport.DataImportHandler
   *version: * 1.0  *description: * Manage data import from databases to
 Solr   *stats: * Status : IDLE
 Documents Processed : 10
 Requests made to DataSource : 1
 Rows Fetched : 10
 Documents Deleted : 0
 Documents Skipped : 0
 Total Documents Processed : 0
 Total Requests made to DataSource : 0
 Total Rows Fetched : 0
 Total Documents Deleted : 0
 Total Documents Skipped : 0
 handlerStart : 1293831460260
 requests : 2





Re: solr newbie: Diagnose why DataImportHandler DIH not saving documents

2010-12-31 Thread Ahmet Arslan
It seems that with debug=on there is a hard coded default rows=10.

http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=onechoParams=allrows=50

returns  Added/Updated: 50 documents. Deleted 0 documents.

It seems that debug parameter is related to /solr/core0/admin/dataimport.jsp 
page.

Don't know exact purpose of debug parameter but, can't you just ignore it and 
use

http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-import


--- On Sat, 1/1/11, Stephen Boesch java...@gmail.com wrote:

 From: Stephen Boesch java...@gmail.com
 Subject: Re: solr newbie: Diagnose why DataImportHandler DIH not saving 
 documents
 To: solr-user@lucene.apache.org
 Date: Saturday, January 1, 2011, 3:09 AM
 one little extra piece of info: part
 of the stats page got omitted - notably
 the number of errors was reported as 0.
 
 errors : 0
 timeouts : 0
 totalTime : 1963
 avgTimePerRequest : 981.5
 avgRequestsPerSecond : 0.0011371888
 
 
 2010/12/31 Stephen Boesch java...@gmail.com
 
  I am asking for a full DataImport via a url.  It
 seems to be partially
   happy with the request - with debug=on I can see
 it saying that 10
  documents were indexed.  The backend however
 realizes there are actually 440
  records available for the query.
 
  Not sure why only 10 records were selected and then
 why even those 10
  records are not stored.
 
 
  Here is the obfuscated url used for invoking the
 DataImport:
 
 
 mySolrHost:8983/solr/core0/dataimport?command=full-importdebug=onhttp://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=on
 
 
  Here is the output:  looks reasonable for the 10
 records it does find:
  notice it says *added/updated 10 documents*
 
  0360db-data-config.xmlfull-importdebugBrad is testing
 
 thisjava.math.BigDecimal:1java.math.BigDecimal:15000947 Wood
 Duck
 
 Lanejava.math.BigDecimal:3java.math.BigDecimal:15002Stanford
 Quad
 
 Sculpturejava.math.BigDecimal:3java.math.BigDecimal:15200Apple
 Store - Palo
 
 Altojava.math.BigDecimal:3java.math.BigDecimal:15201Fox
 
 Theaterjava.math.BigDecimal:3java.math.BigDecimal:15220java.math.BigDecimal:3java.math.BigDecimal:15222Knowtate
  promojava.math.BigDecimal:4welcome to
 Knowtatejava.math.BigDecimal:16163The
  Green Dragon
 Tavernjava.math.BigDecimal:5java.math.BigDecimal:15020The
 All
  New Infiniti
 Mjava.math.BigDecimal:5Introjava.math.BigDecimal:15100The
 All
  New Infiniti Mjava.math.BigDecimal:5To hear current
  specialsjava.math.BigDecimal:15100idleConfiguration
 Re-loaded
  sucessfully11002010-12-31 16:45:11Indexing completed.
 *Added/Updated: 10
  documents. *Deleted 0 documents.100:0:0.331This
 response format is
  experimental. It is likely to change in the future.
 
 
  But when I go to the Admin screen, it tells
 me   Documents Processed: 10
  *Total Documents Processed 0*
  *
  *
  So what is difference between Documents and Total
 Documents ??  Note that
  there is presently *no *data in the indexes.
 
 
 mySolrHost:8983/solr/core0/admin/stats.jsphttp://knowtate.servehttp.com:8983/solr/core0/admin/
 
  *name: * /dataimport   *class: *
 org.apache.solr.handler.dataimport.DataImportHandler
    *version: * 1.0  *description: *
 Manage data import from databases to
  Solr   *stats: * Status : IDLE
  Documents Processed : 10
  Requests made to DataSource : 1
  Rows Fetched : 10
  Documents Deleted : 0
  Documents Skipped : 0
  Total Documents Processed : 0
  Total Requests made to DataSource : 0
  Total Rows Fetched : 0
  Total Documents Deleted : 0
  Total Documents Skipped : 0
  handlerStart : 1293831460260
  requests : 2
 
 
 
 


  


Re: solr newbie: Diagnose why DataImportHandler DIH not saving documents

2010-12-31 Thread Stephen Boesch
sure I'll try that.

2010/12/31 Ahmet Arslan iori...@yahoo.com

 It seems that with debug=on there is a hard coded default rows=10.


 http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=onechoParams=allrows=50

 returns  Added/Updated: 50 documents. Deleted 0 documents.

 It seems that debug parameter is related to
 /solr/core0/admin/dataimport.jsp page.

 Don't know exact purpose of debug parameter but, can't you just ignore it
 and use


 http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-import


 --- On Sat, 1/1/11, Stephen Boesch java...@gmail.com wrote:

  From: Stephen Boesch java...@gmail.com
  Subject: Re: solr newbie: Diagnose why DataImportHandler DIH not saving
 documents
  To: solr-user@lucene.apache.org
  Date: Saturday, January 1, 2011, 3:09 AM
  one little extra piece of info: part
  of the stats page got omitted - notably
  the number of errors was reported as 0.
 
  errors : 0
  timeouts : 0
  totalTime : 1963
  avgTimePerRequest : 981.5
  avgRequestsPerSecond : 0.0011371888
 
 
  2010/12/31 Stephen Boesch java...@gmail.com
 
   I am asking for a full DataImport via a url.  It
  seems to be partially
happy with the request - with debug=on I can see
  it saying that 10
   documents were indexed.  The backend however
  realizes there are actually 440
   records available for the query.
  
   Not sure why only 10 records were selected and then
  why even those 10
   records are not stored.
  
  
   Here is the obfuscated url used for invoking the
  DataImport:
  
  
  mySolrHost:8983/solr/core0/dataimport?command=full-importdebug=on
 http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=on
 
  
  
   Here is the output:  looks reasonable for the 10
  records it does find:
   notice it says *added/updated 10 documents*
  
   0360db-data-config.xmlfull-importdebugBrad is testing
  
  thisjava.math.BigDecimal:1java.math.BigDecimal:15000947 Wood
  Duck
  
  Lanejava.math.BigDecimal:3java.math.BigDecimal:15002Stanford
  Quad
  
  Sculpturejava.math.BigDecimal:3java.math.BigDecimal:15200Apple
  Store - Palo
  
  Altojava.math.BigDecimal:3java.math.BigDecimal:15201Fox
  
 
 Theaterjava.math.BigDecimal:3java.math.BigDecimal:15220java.math.BigDecimal:3java.math.BigDecimal:15222Knowtate
   promojava.math.BigDecimal:4welcome to
  Knowtatejava.math.BigDecimal:16163The
   Green Dragon
  Tavernjava.math.BigDecimal:5java.math.BigDecimal:15020The
  All
   New Infiniti
  Mjava.math.BigDecimal:5Introjava.math.BigDecimal:15100The
  All
   New Infiniti Mjava.math.BigDecimal:5To hear current
   specialsjava.math.BigDecimal:15100idleConfiguration
  Re-loaded
   sucessfully11002010-12-31 16:45:11Indexing completed.
  *Added/Updated: 10
   documents. *Deleted 0 documents.100:0:0.331This
  response format is
   experimental. It is likely to change in the future.
  
  
   But when I go to the Admin screen, it tells
  me   Documents Processed: 10
   *Total Documents Processed 0*
   *
   *
   So what is difference between Documents and Total
  Documents ??  Note that
   there is presently *no *data in the indexes.
  
  
  mySolrHost:8983/solr/core0/admin/stats.jsp
 http://knowtate.servehttp.com:8983/solr/core0/admin/
  
   *name: * /dataimport   *class: *
  org.apache.solr.handler.dataimport.DataImportHandler
 *version: * 1.0  *description: *
  Manage data import from databases to
   Solr   *stats: * Status : IDLE
   Documents Processed : 10
   Requests made to DataSource : 1
   Rows Fetched : 10
   Documents Deleted : 0
   Documents Skipped : 0
   Total Documents Processed : 0
   Total Requests made to DataSource : 0
   Total Rows Fetched : 0
   Total Documents Deleted : 0
   Total Documents Skipped : 0
   handlerStart : 1293831460260
   requests : 2
  
  
  
 






Re: solr newbie: Diagnose why DataImportHandler DIH not saving documents

2010-12-31 Thread Stephen Boesch
Yes that fixed the problem.  interesting.. usually think setting debug just
changes the verbosity level.. in this case caused docs not to be processed.

02db-data-config.xmlfull-importidle144002010-12-31 17:45:03Indexing
completed. Added/Updated: 440 documents. Deleted 0 documents.2010-12-31
17:45:032010-12-31 17:45:034400:0:0.258This response format is experimental.
It is likely to change in the future.

Now I am seeing the full 440 docs being processed.
cool!

*ame: */dataimport  *class:
*org.apache.solr.handler.dataimport.DataImportHandler
 *version: *1.0  *description: *Manage data import from databases to Solr  *
stats: *Status : IDLE
Documents Processed : 440
Requests made to DataSource : 1
Rows Fetched : 440
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 880
Total Requests made to DataSource : 2
Total Rows Fetched : 880
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1293831460260
requests : 35
errors : 0
timeouts : 0
totalTime : 3170
avgTimePerRequest : 90.57143
avgRequestsPerSecond : 0.008557899

2010/12/31 Stephen Boesch java...@gmail.com

 sure I'll try that.

 2010/12/31 Ahmet Arslan iori...@yahoo.com

 It seems that with debug=on there is a hard coded default rows=10.


 http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=onechoParams=allrows=50

 returns  Added/Updated: 50 documents. Deleted 0 documents.

 It seems that debug parameter is related to
 /solr/core0/admin/dataimport.jsp page.

 Don't know exact purpose of debug parameter but, can't you just ignore it
 and use


 http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-import


 --- On Sat, 1/1/11, Stephen Boesch java...@gmail.com wrote:

  From: Stephen Boesch java...@gmail.com
  Subject: Re: solr newbie: Diagnose why DataImportHandler DIH not saving
 documents
  To: solr-user@lucene.apache.org
  Date: Saturday, January 1, 2011, 3:09 AM
  one little extra piece of info: part
  of the stats page got omitted - notably
  the number of errors was reported as 0.
 
  errors : 0
  timeouts : 0
  totalTime : 1963
  avgTimePerRequest : 981.5
  avgRequestsPerSecond : 0.0011371888
 
 
  2010/12/31 Stephen Boesch java...@gmail.com
 
   I am asking for a full DataImport via a url.  It
  seems to be partially
happy with the request - with debug=on I can see
  it saying that 10
   documents were indexed.  The backend however
  realizes there are actually 440
   records available for the query.
  
   Not sure why only 10 records were selected and then
  why even those 10
   records are not stored.
  
  
   Here is the obfuscated url used for invoking the
  DataImport:
  
  
  mySolrHost:8983/solr/core0/dataimport?command=full-importdebug=on
 http://knowtate.servehttp.com:8983/solr/core0/dataimport?command=full-importdebug=on
 
  
  
   Here is the output:  looks reasonable for the 10
  records it does find:
   notice it says *added/updated 10 documents*
  
   0360db-data-config.xmlfull-importdebugBrad is testing
  
  thisjava.math.BigDecimal:1java.math.BigDecimal:15000947 Wood
  Duck
  
  Lanejava.math.BigDecimal:3java.math.BigDecimal:15002Stanford
  Quad
  
  Sculpturejava.math.BigDecimal:3java.math.BigDecimal:15200Apple
  Store - Palo
  
  Altojava.math.BigDecimal:3java.math.BigDecimal:15201Fox
  
 
 Theaterjava.math.BigDecimal:3java.math.BigDecimal:15220java.math.BigDecimal:3java.math.BigDecimal:15222Knowtate
   promojava.math.BigDecimal:4welcome to
  Knowtatejava.math.BigDecimal:16163The
   Green Dragon
  Tavernjava.math.BigDecimal:5java.math.BigDecimal:15020The
  All
   New Infiniti
  Mjava.math.BigDecimal:5Introjava.math.BigDecimal:15100The
  All
   New Infiniti Mjava.math.BigDecimal:5To hear current
   specialsjava.math.BigDecimal:15100idleConfiguration
  Re-loaded
   sucessfully11002010-12-31 16:45:11Indexing completed.
  *Added/Updated: 10
   documents. *Deleted 0 documents.100:0:0.331This
  response format is
   experimental. It is likely to change in the future.
  
  
   But when I go to the Admin screen, it tells
  me   Documents Processed: 10
   *Total Documents Processed 0*
   *
   *
   So what is difference between Documents and Total
  Documents ??  Note that
   there is presently *no *data in the indexes.
  
  
  mySolrHost:8983/solr/core0/admin/stats.jsp
 http://knowtate.servehttp.com:8983/solr/core0/admin/
  
   *name: * /dataimport   *class: *
  org.apache.solr.handler.dataimport.DataImportHandler
 *version: * 1.0  *description: *
  Manage data import from databases to
   Solr   *stats: * Status : IDLE
   Documents Processed : 10
   Requests made to DataSource : 1
   Rows Fetched : 10
   Documents Deleted : 0
   Documents Skipped : 0
   Total Documents Processed : 0
   Total Requests made to DataSource : 0
   Total Rows Fetched : 0
   Total Documents Deleted : 0
   Total Documents Skipped : 0
   handlerStart : 1293831460260
   requests : 2
  
  
  
 







solr benchmarks

2010-12-31 Thread Tri Nguyen
Hi,
 
I remember going through some page that had graphs of response times based on 
index size for solr.
 
Anyone know of such pages?
 
Internally, we have some requirements for response times and I'm trying to 
figure out when to shard the index.
 
Thanks,
 
Tri

Re: solr benchmarks

2010-12-31 Thread François Schiettecatte
I would shard the index so that each shard is no larger than the memory of the 
machine it sits on, that way your entire index will be in memory all the time. 
When I was at Feedster (I wrote the search engine), the rule of thumb I had was 
to have 14GB of index on a 16GB machine.

François

On Dec 31, 2010, at 9:06 PM, Tri Nguyen wrote:

 Hi,
  
 I remember going through some page that had graphs of response times based on 
 index size for solr.
  
 Anyone know of such pages?
  
 Internally, we have some requirements for response times and I'm trying to 
 figure out when to shard the index.
  
 Thanks,
  
 Tri