Re: Error with bin/optimize and multiple solr webapps
On 3/7/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: Oops, my bad I didn't see either 186 or 187 before entering 188. :-) I have closed SOLR-186 and SOLR-187 as duplicates, please add relevant info to SOLR-188 if needed. -Bertrand
Re: Hierarchical Facets
Hmm ... I had a brain storm. Could I do something like this: collectionDir1/Subdir1/SubSubDir1/collection Then query collection:Dir1/Subdir1 and get the facets on collection at that point to see all of the subsubdirectories? Is their any better method? Andrew Andrew Nagy wrote: I am running into a stumbling block and can only find a way to solve the problem with some sort of hierarchical faceting system. I am in the process of moving my records from eXist (http://exist.sf.net) to Solr, but the problem is with the lack of a directory structure that exist has. I figure I could easily solve this problem by setting a field called collection and then faceting on that field to get my list of directories however this will not allow me to do subdirectories. Any ideas on how to implement such a thing in solr? Thanks! Andrew
Re: [2] SQL Update
: It would be useful to have a solr setting for stripping the dynamic field : suffix/prefix on index field name to get back the original field name. Does : it make sense? sorry, i'm really not following this talk of stripping the dynamic field name to get back the original field name ... what does that mean? it seems like what you are describing is having a schema consisting entirely of dynamic fields where teh suffix determines hte datatype, but you don't wnat to have to use that suffix when interacting with Solr ... but if i've got two dynamicFields *_i and *_s and you refer to user in your query, how is Solr suppose to know wether you want user_i or user_s If you want people do be able to refer to the field user and you want it to be an integer field, why not definie it as an explicit field instead of a dynamic field? -Hoss
Re: improve performance after commit
: back in just now. Here's an example trying to warm using a sort on : field name subject. I tried query of : allMessageContent:trying;subject+asc as well as : allMessageContent:trying;subject (without +asc) - either way when expressing params in XML (either as init params for a request handler, or in a QuerySenderListener the params don't need to be URL escaped ... they just need to be XML escaped, try something like... listener event=newSearcher class=solr.QuerySenderListener arr name=queries lst str name=qallMessageContent:test; subject asc/str str name=start0/str str name=rows10/str /lst /arr /listener -Hoss
Re: Hierarchical Facets
Assuming I'm understanding your question correctly, something like that would probably work just fine -- just do prefix searches on your directories when you want to find all files in that directory or any decendent directory, and use facet counds to see the list of decendent directories and the number of files under each that match. : Date: Wed, 07 Mar 2007 10:11:54 -0500 : From: Andrew Nagy [EMAIL PROTECTED] : Reply-To: solr-user@lucene.apache.org : To: Andrew Nagy [EMAIL PROTECTED] : Cc: solr-user@lucene.apache.org : Subject: Re: Hierarchical Facets : : Hmm ... I had a brain storm. : : Could I do something like this: : collectionDir1/Subdir1/SubSubDir1/collection : : Then query collection:Dir1/Subdir1 and get the facets on collection at : that point to see all of the subsubdirectories? : : Is their any better method? : : Andrew : : Andrew Nagy wrote: : I am running into a stumbling block and can only find a way to solve : the problem with some sort of hierarchical faceting system. I am in : the process of moving my records from eXist (http://exist.sf.net) to : Solr, but the problem is with the lack of a directory structure that : exist has. I figure I could easily solve this problem by setting a : field called collection and then faceting on that field to get my : list of directories however this will not allow me to do : subdirectories. : : Any ideas on how to implement such a thing in solr? : : Thanks! : Andrew : : -Hoss
Re: [2] Highlighting problems with HTML tagged fields
It is tracked in http://issues.apache.org/jira/browse/SOLR-42 ...there are currently no patches. : Date: Tue, 6 Mar 2007 15:04:25 -0800 (PST) : From: nick19701 [EMAIL PROTECTED] : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject: Re: [2] Highlighting problems with HTML tagged fields : : : : Yonik Seeley wrote: : : HTMLStripWhitespaceTokenizerFactory works in two phases... : HTMLStripReader removes the HTML and passes the result to : WhitespaceTokenizer... at that point, Tokens are generated, but the : offsets will correspond to the text after HTML removal, not before. : : I did it this way so that HTMLStripReader could go before any : tokenizer (like StandardTokenizer). : : Can you open a JIRA bug for this? The fix would be a special version : of HTMLStripReader integrated with a WhitespaceTokenizer to keep : offsets correct. : : -Yonik : : : Is there a fix for this problem? : : my solr is dated on 12/17/2006. HTMLStripWhitespaceTokenizerFactory + : highlighting still : doesn't work. All the wrong items are highlighted. : -- : View this message in context: http://www.nabble.com/Highlighting-problems-with-HTML-tagged-fields-tf2017260.html#a9343253 : Sent from the Solr - User mailing list archive at Nabble.com. : -Hoss
Hierarchical Facets
I am running into a stumbling block and can only find a way to solve the problem with some sort of hierarchical faceting system. I am in the process of moving my records from eXist (http://exist.sf.net) to Solr, but the problem is with the lack of a directory structure that exist has. I figure I could easily solve this problem by setting a field called collection and then faceting on that field to get my list of directories however this will not allow me to do subdirectories. Any ideas on how to implement such a thing in solr? Thanks! Andrew
Re: dynamicField Scaling
On Mar 7, 2007, at 2:17 PM, Mike Klaas wrote: On 3/7/07, mark angelillo [EMAIL PROTECTED] wrote: Hello, I've got a Solr index running and I want to use a dynamicField to store n different sorting fields. The field that is used to actually sort the results will be determined by the application that is querying the index. I'm wondering if anyone has done something similar to this, or if anyone has an idea of how Solr will perform as the number n of sorting fields grows larger. Is there a way to make sure this doesn't start to slow the index down? Is there any information out there about the number of dynamicFields that can be declared in this way before the entire index suffers? Is there such a limit? It's not realy about the number of dynamic fields. The key variable is the number of sort fields. To sort efficiently, solr needs to maintain a cache of field values. This consumes memory per-field on the order of D x S + U where D is the document count, S is the the size of the data type (eg. 4bytes for ints, 8 bytes for doubles, 4/8 bytes for anything else [pointers]), and U is the cumulative size of the unique field values (if sorting on a non-primitive type, like Strings). If you have sufficient memory to store this data for each field you are sorting on, you shouldn't have any problems. best, -Mike Okay, makes sense. Thanks, Mark
xml generation for SOLR
hai, Yes the Xml formats is understood but there is an issue to generate these xmls from a data source. These XML feild tags doesnot contain the same start tags and end tags. like field name=catsoftware/field and standerd xml writers have xml generated as the same start and end tags. in SOLR xml start tag = field name=cat end tag = /field can you adivise anything on this please. regards, aditya
Installation woes
Hello, I am trying to install another copy of solr on a server. I have done this many times before, but am running into errors now and I am not sure what is causing them. I unzipped a copy of 1.1.0 and placed the .war file into tomcat. Then I created the solr directory with my bin, data, conf directories. When I restart tomcat (from the solr directory) and go to the Solr admin website everything looks fine. But, when I pull up the statistics page, I only have information in the Query Handlers box, the rest are empty. When I click on ping, I see a 500 error from tomcat. What am I forgetting to do? Thanks Andrew
Re: Installation woes
On 3/7/07, Andrew Nagy [EMAIL PROTECTED] wrote: Hello, I am trying to install another copy of solr on a server. I have done this many times before, but am running into errors now and I am not sure what is causing them. I unzipped a copy of 1.1.0 and placed the .war file into tomcat. Then I created the solr directory with my bin, data, conf directories. When I restart tomcat (from the solr directory) and go to the Solr admin website everything looks fine. But, when I pull up the statistics page, I only have information in the Query Handlers box, the rest are empty. When I click on ping, I see a 500 error from tomcat. What am I forgetting to do? Did you point solr to the solr_home (containing your bin, data, and conf directories)? Or do you rely on the default of $CWD/solr? If that's not it, check all the tomcat log files for the first exception you see. -Yonik
RE: Installation woes
I had a problem like that when I blew away an index by deleting the index directory instead of its parent the data directory; it seemed that if Solr saw the data directory, it assumed the index was there. Removing the data directory and letting Solr create it seemed to fix the problem. (Or maybe something else got fixed at the same time, hard to say). Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 07, 2007 1:12 PM To: solr-user@lucene.apache.org Subject: Installation woes Hello, I am trying to install another copy of solr on a server. I have done this many times before, but am running into errors now and I am not sure what is causing them. I unzipped a copy of 1.1.0 and placed the .war file into tomcat. Then I created the solr directory with my bin, data, conf directories. When I restart tomcat (from the solr directory) and go to the Solr admin website everything looks fine. But, when I pull up the statistics page, I only have information in the Query Handlers box, the rest are empty. When I click on ping, I see a 500 error from tomcat. What am I forgetting to do? Thanks Andrew
Re: Installation woes
Argh! Thanks Yonik for pointing out the log files, duh! I had a malformed line in my schema.xml. Nice feature to add down the line, although I know there is a lot of work going into the admin interface so who knows if it is already thought of. Schema Debugger? Maybe one day I will dig into the code guts and try to contribute. Andrew Yonik Seeley wrote: On 3/7/07, Andrew Nagy [EMAIL PROTECTED] wrote: Hello, I am trying to install another copy of solr on a server. I have done this many times before, but am running into errors now and I am not sure what is causing them. I unzipped a copy of 1.1.0 and placed the .war file into tomcat. Then I created the solr directory with my bin, data, conf directories. When I restart tomcat (from the solr directory) and go to the Solr admin website everything looks fine. But, when I pull up the statistics page, I only have information in the Query Handlers box, the rest are empty. When I click on ping, I see a 500 error from tomcat. What am I forgetting to do? Did you point solr to the solr_home (containing your bin, data, and conf directories)? Or do you rely on the default of $CWD/solr? If that's not it, check all the tomcat log files for the first exception you see. -Yonik
Re: xml generation for SOLR
: like field name=catsoftware/field : : and standerd xml writers have xml generated as the same start and end tags. : : in SOLR xml : start tag = field name=cat : end tag = /field that tag is the same, the tag is field ... the field tag has a mandatory attribute which is name While it's certianly true that some people discourage use of XML attributes when designing XML schema,s it doesn't change the fact that having attributes is a very legal part of XML. a quick google search for xml attribute turns up quite a few pages that you may find useful, here's just a few for starters... http://java.sun.com/webservices/jaxp/dist/1.1/docs/tutorial/overview/1_xml.html http://www.xmlnews.org/docs/xml-basics.html#attributes http://www.expertrating.com/courseware/XMLCourse/XML-Elements-Attributes-4.asp -Hoss
Solr on Tomcat 6.0.10?
Is anyone running Solr on Tomcat 6.0.10? Any issues? I searched the archives and didn't see anything. wunder -- Walter Underwood Search Guru, Netflix
Re: Installation woes
On 3/7/07, Andrew Nagy [EMAIL PROTECTED] wrote: Argh! Thanks Yonik for pointing out the log files, duh! I had a malformed line in my schema.xml. Nice feature to add down the line, although I know there is a lot of work going into the admin interface so who knows if it is already thought of. Schema Debugger? Maybe one day I will dig into the code guts and try to contribute. I recently posted: https://issues.apache.org/jira/browse/SOLR-179 It stops everything and displays the errors on every url if there are configuration errors.
Solr and Multiple Index Partitions
Hello there, Howdy. I was wondering if there is a way to configure one Solr instance to search multiple Index partitions? I read the wiki and found the entry in SolrConfig.xml: dataDir/var/data/solr/dataDir Can I have mutiple directories? Comma separated? Any help is greatly appreciated. -- Thanks, Venkatesh Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away. - Antoine de Saint-Exupéry
Re: Solr and Multiple Index Partitions
Solr looks at one index - If you want to look at multiple indexes, you need multiple solr instances running. Check the wiki for how to set that up: http://wiki.apache.org/solr/SolrJetty (the resin and tomcat pages have something similar) On 3/7/07, Venkatesh Seetharam [EMAIL PROTECTED] wrote: Hello there, Howdy. I was wondering if there is a way to configure one Solr instance to search multiple Index partitions? I read the wiki and found the entry in SolrConfig.xml: dataDir/var/data/solr/dataDir Can I have mutiple directories? Comma separated? Any help is greatly appreciated. -- Thanks, Venkatesh Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away. - Antoine de Saint-Exupéry
Re: Solr and Multiple Index Partitions
Yes, I'm implementing federated search. I do have N partitions of indexes built and I'd like to have mutiple Solr instances in a cluster each serving atleast 2 partitions. I was wondering if I could somehow find a way to make Solr work with atleast 2 partitions. It looks like I need to have smaller number of partitions. Thanks, Venkatesh On 3/7/07, Bradley Burke [EMAIL PROTECTED] wrote: Venkatesh, Are you perhapes talking about Federated Searching ( http://wiki.apache.org/solr/FederatedSearch). I am new to Solr but this was a feature I was looking far as well. I do not think its built into Solr at the momment. Regards, Brad On 3/7/07, Venkatesh Seetharam [EMAIL PROTECTED] wrote: Thanks Ryan for your inputs. If I'm not using Solr webapp but wrapping Solr in plain Java, is there any way that I could get Solr to work with multiple index partitions? Venkatesh On 3/7/07, Ryan McKinley [EMAIL PROTECTED] wrote: Solr looks at one index - If you want to look at multiple indexes, you need multiple solr instances running. Check the wiki for how to set that up: http://wiki.apache.org/solr/SolrJetty (the resin and tomcat pages have something similar) On 3/7/07, Venkatesh Seetharam [EMAIL PROTECTED] wrote: Hello there, Howdy. I was wondering if there is a way to configure one Solr instance to search multiple Index partitions? I read the wiki and found the entry in SolrConfig.xml: dataDir/var/data/solr/dataDir Can I have mutiple directories? Comma separated? Any help is greatly appreciated. -- Thanks, Venkatesh Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away. - Antoine de Saint-Exupéry
Re: Solr and Multiple Index Partitions
Thanks Ryan for your insight. I do not wish to change Solr. Although it seems weird, just adding a type field makes it possible I'm not working with a DB but since we are indexing a huge vault of XML documents, I use Hadoop + Lucene for indexing which finally generates N number of partitions, I was thinking to use Solr as the searcher and will have a broker which merges results from individual Solr searchers. I do not know if I can ask another question here or start a new thread. I use a custom Analyzer which extends Lucene's StandardAnalyzer. When I configured Solr to use this one, It throws an exception RuntimeException(Can't set positionIncrementGap on custom analyzer + analyzer.getClass()). Do I need to extend a specific Analyzer for it to work with Solr? Venkatesh On 3/7/07, Ryan McKinley [EMAIL PROTECTED] wrote: As it is now... I don't think so. SolrCore is a static singleton class -- without some serious reworking, i think there is only one instance per jvm. I think getting rid of the static singleton should go on the long term TODO list, but that doesn't help you now. Although it seems weird, just adding a type field makes it possible to put things that seem like they need their own database into the same index - you select the 'type' you want by adding +type:mytype to your query On 3/7/07, Venkatesh Seetharam [EMAIL PROTECTED] wrote: Thanks Ryan for your inputs. If I'm not using Solr webapp but wrapping Solr in plain Java, is there any way that I could get Solr to work with multiple index partitions? Venkatesh On 3/7/07, Ryan McKinley [EMAIL PROTECTED] wrote: Solr looks at one index - If you want to look at multiple indexes, you need multiple solr instances running. Check the wiki for how to set that up: http://wiki.apache.org/solr/SolrJetty (the resin and tomcat pages have something similar) On 3/7/07, Venkatesh Seetharam [EMAIL PROTECTED] wrote: Hello there, Howdy. I was wondering if there is a way to configure one Solr instance to search multiple Index partitions? I read the wiki and found the entry in SolrConfig.xml: dataDir/var/data/solr/dataDir Can I have mutiple directories? Comma separated? Any help is greatly appreciated. -- Thanks, Venkatesh Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away. - Antoine de Saint-Exupéry
Re: [2] Highlighting problems with HTML tagged fields
Chris Hostetter wrote: It is tracked in http://issues.apache.org/jira/browse/SOLR-42 ...there are currently no patches. The suggested fix from Mirko seems very simple. Hopefull a patch will be applied very soon. In the meantime, I'll use my backup solution: http://fucoder.com/code/se-hilite/ http://fucoder.com/code/se-hilite/ -- View this message in context: http://www.nabble.com/Highlighting-problems-with-HTML-tagged-fields-tf2017260.html#a9363720 Sent from the Solr - User mailing list archive at Nabble.com.
Re: [2] SQL Update
: I wanted to add data from relational database tables. : To avoid defining each and every table column name in the schema, I thought : I'll append a suffix to the field name depending on it's type. which is fine and dandy for when you index the data, mapping your string database column user, and your int database column id to the solr dynamic fields user_string and id_int ...and i suppose you could make a customized ResponseWriter that when writing out documents striped off any suffixes it could tell came from dynamicFields so the response docs contained str name=user and int name=id ... but when parsing the query string your clients send, and they ask for user:42 how would the request handler know that it shoudl rewrite that to user_string:42 and not user_int:42 ? -Hoss
Re: [2] Highlighting problems with HTML tagged fields
Chris Hostetter wrote: patches for issues can't be applied until someone who cares about them write them and contribute them for committers to consider/apply :) it seems I'm one of the very few people who care about this feature :) Unfortunately my daily languages are c++ and c#. I only know a little bit Java. Otherwise I'll contribute. -- View this message in context: http://www.nabble.com/Highlighting-problems-with-HTML-tagged-fields-tf2017260.html#a9365098 Sent from the Solr - User mailing list archive at Nabble.com.
synonym filter fix
About this synonym filter fix: http://issues.apache.org/jira/browse/SOLR-167 http://issues.apache.org/jira/browse/SOLR-167 I tried today's solr build. It seems the verbose analysis of the index analyzer still has the same symptom. should I try another build? Or maybe my tomcat is not using the latest build? -- View this message in context: http://www.nabble.com/synonym-filter-fix-tf3365934.html#a9365249 Sent from the Solr - User mailing list archive at Nabble.com.
Re: synonym filter fix
On 3/7/07, nick19701 [EMAIL PROTECTED] wrote:
About this synonym filter fix:
http://issues.apache.org/jira/browse/SOLR-167
http://issues.apache.org/jira/browse/SOLR-167
I tried today's solr build. It seems the verbose analysis of the index
analyzer still has the same symptom.
should I try another build? Or maybe my tomcat is not using the latest
build?
Using the latest solr trunk, the error I reproduced before seems fixed
(see below). Note that if you are using term vectors with
highlighting, you'll have to reindex.
org.apache.solr.analysis.SynonymFilterFactory {expand=true,
ignoreCase=true, synonyms=synonyms.txt}
term position 1 2 3 4 5 6 7 8
9 10 11 12 13
term text bestbuy - AcerAspire AS5610-2273 -
$599. Windows vista, 1 GB RAM
bestbuy gib
bb gigabyte
gigabytes
term type wordwordwordwordwordwordwordword
wordwordwordwordword
wordword
wordword
word
source start,end
0,8 0,8 9,1011,15 16,22 23,34 35,36 37,42 43,50
51,57 58,59 60,62 63,66
0,8 60,62
0,8 60,62
60,62
Re[2]: Solr and Multiple Index Partitions
Selecting by type will do the job. But I suppose it sacrifice performance because having multiple document types in the same index will render a larger index. Is it bad? -- Best regards, Jack Wednesday, March 7, 2007, 2:15:14 PM, you wrote: As it is now... I don't think so. SolrCore is a static singleton class -- without some serious reworking, i think there is only one instance per jvm. I think getting rid of the static singleton should go on the long term TODO list, but that doesn't help you now. Although it seems weird, just adding a type field makes it possible to put things that seem like they need their own database into the same index - you select the 'type' you want by adding +type:mytype to your query __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: making an in-order query
id:A id:B id:C id:D *usually* works, but I have seen D appear first in the results for certain queries. Is there a query I can do or a better way to accomplish this? It's a bit of a hack, but you could use boosts to order the docs: id:A^4 id:B^3 id:C^2 id:D^1 Gorgeous! Does the job admirably. Thanks Yonik
