Authentication for REST-RPC Webservices
Hi, Has anybody successfully called a REST-RPC Webservice for basic authentication. I would like to which is better one REST-RPC or REST with SOAP/WSDL and why? Regards Dilip
RE: Authentication for REST-RPC Webservices
Hi, To add to my earlier query which would be better a) using REST-RPC or b) using RESTFul Webservices using JAX-WS ? Regards Dilip -Original Message- From: Dilip.TS [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 18, 2007 11:41 AM To: solr-user@lucene.apache.org Subject: Authentication for REST-RPC Webservices Hi, Has anybody successfully called a REST-RPC Webservice for basic authentication. I would like to which is better one REST-RPC or REST with SOAP/WSDL and why? Regards Dilip
Searching items with in the search results with SOLR
Hi, Is it possible to Search items with in the search results using SOLR. If so how? Thanks in advance, Regards, Dilip
Re: Combining Proximity Range search
I am trying my best to figure out the correct way to do this http://www.nabble.com/forum/ViewPost.jtp?post=12456399framed=y I just came across the collapseFilter ( solr 236). I think i will give that a try. Thanks hossman for the reply Bharani hossman wrote: : My document will have a multivalued compound field like : : revision_01012007 : review_02012007 : : i am thinking of a query like comp:type:review date:[02012007 TO : 02282007]~0 your best bet is to change that so revision and review are the names of a field, and do a range search on them as needed. -Hoss -- View this message in context: http://www.nabble.com/Combining-Proximity---Range-search-tf4450179.html#a12751291 Sent from the Solr - User mailing list archive at Nabble.com.
Re: commit, concurrency, full text search
Before a COMMIT is done , lock is obtained and its released after the operation - hence no corruption happens. On 9/18/07, Dilip.TS [EMAIL PROTECTED] wrote: Hi, I have a query, when u try for bulk updates, using autoCommit option (which does commit on regular basis). Suppose if another request tries to update before the record is committed by the first request, a) Dont u think these requests step on each other and corrupt the index files? b) Is there any Lock mechanism involved which prevents this and the rows gets unlocked when commit is done? c) What happens to the lock if there is the commit is not done? does it gets unlocked automatically? Could you elaborate this? Thanks and Regards Dilip -Original Message- From: Mike Klaas [mailto:[EMAIL PROTECTED] Sent: Monday, September 17, 2007 11:29 PM To: solr-user@lucene.apache.org Subject: Re: commit, concurrency, full text search On 16-Sep-07, at 11:23 PM, Dilip.TS wrote: Hi, 1)How does the commit works with multiple requests? Multiple updates? They block while the commit completes. 2)Does SOLR handle the concurrency during updates? It is parallelized as much as possible, yes. 3)Does solr support any thing like, if I enclose the keywords within quotes, then we are searching for exactly those keywords together. Some thing like google does, for example if I enclose like this java programming then it should search for this keyword as a whole instead breaking the phrase apart. Indeed it does. see http://lucene.apache.org/java/docs/queryparsersyntax.html http://wiki.apache.org/solr/SolrQuerySyntax -Mike --
Re: Searching items with in the search results with SOLR
On Sep 18, 2007, at 2:45 AM, Dilip.TS wrote: Is it possible to Search items with in the search results using SOLR. If so how? Simply AND the previous query to the new query, or use the previous query as a filter query (fq=...) parameter. Erik
Search for Java Programming vs Java Programming
Hi, I have the following requirement: When the user searches for the keyword say Java Programming , the user should be shown the results satisfying the condition Java AND Programming. But when he types Java Programming (i.e within double quotes), the user should be shown with the results exactly matching Java Programming as a single word (similar to doing a search on google). The user should be shown with the proper results accordingly. Any help would be highly appreciated. Thanks in advance, Regards, Dilip TS
Re: Search for Java Programming vs Java Programming
On Sep 18, 2007, at 7:14 AM, Dilip.TS wrote: Hi, I have the following requirement: When the user searches for the keyword say Java Programming , the user should be shown the results satisfying the condition Java AND Programming. But when he types Java Programming (i.e within double quotes), the user should be shown with the results exactly matching Java Programming as a single word (similar to doing a search on google). The user should be shown with the proper results accordingly. Any help would be highly appreciated. Dilip - are you seeing results indicating a problem with this in your application? Query syntax is spelled out in more detail here: http://wiki.apache.org/solr/SolrQuerySyntax The one difference you may have is solrQueryParser defaultOperator=OR/ in schema.xml. Change that to AND to satisfy your first requirement. Erik
Customize the way relevancy is calculated
Hi, I have the following requirement. We have an option to sort the search results by relevance. Right now we are getting the search results making an http call If the user chooses that option, the results should be sorted based on the relevance. But the relevance should be calculated in a particular way, for example for a total of 100%, key word match will contribute 35%, availability of image for that product will contribute 15% and so on... The 65% of the relevance can be computed while indexing the document and posted as a field. But the keyword match is a run time score .Is there any way of getting the relevance score as a combination of this 65% and 35%? Any help would be appreciated. Regards Amitha DISCLAIMER: This message (including attachment if any) is confidential and may be privileged. If you have received this message by mistake please notify the sender by return e-mail and delete this message from your system. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited. E-mail may contain viruses. Before opening attachments please check them for viruses and defects. While MindTree Consulting Limited (MindTree) has put in place checks to minimize the risks, MindTree will not be responsible for any viruses or defects or any forwarded attachments emanating either from within MindTree or outside. Please note that e-mails are susceptible to change and MindTree shall not be liable for any improper, untimely or incomplete transmission. MindTree reserves the right to monitor and review the content of all messages sent to or from MindTree e-mail address. Messages sent to or from this e-mail address may be stored on the MindTree e-mail system or else where.
Re: Customize the way relevancy is calculated
On 9/18/07, Amitha Talasila [EMAIL PROTECTED] wrote: The 65% of the relevance can be computed while indexing the document and posted as a field. But the keyword match is a run time score .Is there any way of getting the relevance score as a combination of this 65% and 35%? A FunctionQuery can get you the value of a field to use in a relevancy score. Put that it in a boolean query with the relevanct query and boost each portion to give the correct weight. +text:foo^.65 _val_:scorefield^.35 -Yonik
RE: Search for Java Programming vs Java Programming
Hi, The first requirement works fine for me, but i was unaware of the Phrase Queries for the second one. I found this link very much useful regarding the same. http://wiki.apache.org/solr/SolrRelevancyCookbook Thanks for the help guys, Regards Dilip -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 18, 2007 6:29 PM To: solr-user@lucene.apache.org Subject: Re: Search for Java Programming vs Java Programming On Sep 18, 2007, at 7:14 AM, Dilip.TS wrote: Hi, I have the following requirement: When the user searches for the keyword say Java Programming , the user should be shown the results satisfying the condition Java AND Programming. But when he types Java Programming (i.e within double quotes), the user should be shown with the results exactly matching Java Programming as a single word (similar to doing a search on google). The user should be shown with the proper results accordingly. Any help would be highly appreciated. Dilip - are you seeing results indicating a problem with this in your application? Query syntax is spelled out in more detail here: http://wiki.apache.org/solr/SolrQuerySyntax The one difference you may have is solrQueryParser defaultOperator=OR/ in schema.xml. Change that to AND to satisfy your first requirement. Erik
Re: Search for Java Programming vs Java Programming
[ Kindly do try before posting in the forum or atleast try to look into the wiki and other resources ] nwayz... For the Former: use 'AND' and in the default query For the latter : use phrase queries - enclosing within double quotes should work. On 9/18/07, Dilip.TS [EMAIL PROTECTED] wrote: Hi, I have the following requirement: When the user searches for the keyword say Java Programming , the user should be shown the results satisfying the condition Java AND Programming. But when he types Java Programming (i.e within double quotes), the user should be shown with the results exactly matching Java Programming as a single word (similar to doing a search on google). The user should be shown with the proper results accordingly. Any help would be highly appreciated. Thanks in advance, Regards, Dilip TS --
RE: Triggering snapshooter through web admin interface
: [Wu, Daniel] That sounds great. Do I need to create a JIRA ticket? Sure, JIRA is a great way to track feature requests (since they can be watched and voted on, and if you want to start on an implementation you can attach patches... http://wiki.apache.org/solr/HowToContribute -Hoss
Solr as a transactional data store?
Our program has solved some pretty significant search problems using the Lucene search engine. We're getting ready to take the first step towards an SOA in our back-where we crank out the data that eventually becomes searchable via our website. We're looking for a transactional persistence solution to support our back end entity services. What is the feeling regarding the use of Solr as a transactional data store? Many people here are very fond of Lucene and would like to see Solr employed in that fashion. Is it worth a look over some of the open source native XML databases (eXist, Xindice, Berkeley DB XML)? Has anyone seen this work? Thanks.
Re: commit, concurrency, full text search
: I have a query, when u try for bulk updates, using autoCommit option (which : does commit on regular basis). There is alot of complexity going on when dealing with concurrent updates -- some of it at the Lucene level, some at the Solr level. if you really want to udnerstand the details, I suggest you start by reading the Lucene-In-Action book, then once you understand the Low level Lucene issues, then take a look at the code to see the way Solr manages it's IndexReader and IndexWriter. Or just take my word for it that Solr takes care of the concurrancy issues for you. multiple updates/commits may block, and errors external to Solr (ie: OutOfMemory, full disk, hardware failure, power failure, rouge processes writing to the index files, etc...) may corrupt your index or leave stale lock files lying arround -- but Solr is going to keep it's house in order. -Hoss
RE: Searching items with in the search results with SOLR
Question: if it is a filter query, it will be cached in the filter query cache? Follow-on questions if this is true: Is this the full results of the filter query? What exactly is cached? Thanks, Lance Norskog -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 18, 2007 3:37 AM To: solr-user@lucene.apache.org Subject: Re: Searching items with in the search results with SOLR On Sep 18, 2007, at 2:45 AM, Dilip.TS wrote: Is it possible to Search items with in the search results using SOLR. If so how? Simply AND the previous query to the new query, or use the previous query as a filter query (fq=...) parameter. Erik
DisMax queries referencing undefined fields
Hi, I noticed that the field list (fl) parameter ignores field names that it cannot locate, while the query fields (qf) parameter throws an exception when fields cannot be located. Is there any way to override this behavior and have qf also ignore fields it cannot find? This would be pretty helpful for us, as we're going to have a large number of dynamic, user-specific fields defined in our schema. These fields will have canonical name formats (e.g. userid-comment), but they may not be defined for every document. In fact, some fields may be defined for no documents, which I gather would be the ones that would throw exceptions. It would be nice to provide solr a set of fields that could be searched and have it use the subset of those fields that exist. Thanks in advance for any help, Doug
pluggable functions
I see Yonik recently opened an issue in JIRA to track the addition of pluggable functions (https://issues.apache.org/jira/browse/SOLR-356). Any chance this will be implemented soon? It would save users like me from having to hack the Solr source or write custom request handlers for trivial additions (e.g., adding a distance function), not to mention changes to downstream dependencies (e.g., solr-ruby). Perhaps a reflection-based approach would do the trick? - Jon
Re: pluggable functions
Hi - I'm not sure what you mean by a reflection based approach, but I've been thinking about doing this for a bit, since we needed it, too. I'd just thought about listing class names in the config file. The functions would probably need to extend a subclass of ValueSource which will handle argument parsing for the function, so you won't need to hard code the parsing in a VSParser subclass. I think this might simplify the existing code a bit. You might have to do a bit of reflection to instantiate the function. Did you have an alternate approach in mind? Are there any other things this would need to do? Is anyone else working on this? Tom On 9/18/07, Jon Pierce [EMAIL PROTECTED] wrote: I see Yonik recently opened an issue in JIRA to track the addition of pluggable functions (https://issues.apache.org/jira/browse/SOLR-356). Any chance this will be implemented soon? It would save users like me from having to hack the Solr source or write custom request handlers for trivial additions (e.g., adding a distance function), not to mention changes to downstream dependencies (e.g., solr-ruby). Perhaps a reflection-based approach would do the trick? - Jon
Re: pluggable functions
On 9/18/07, Tom Hill [EMAIL PROTECTED] wrote: Hi - I'm not sure what you mean by a reflection based approach, but I've been thinking about doing this for a bit, since we needed it, too. Reflection could be used to look up and invoke the constructor with appropriately-typed arguments. If we assume only primitive types and ValueSources are used, I don't think it would be too hard to craft a drop-in replacement that works with existing implementations. In any case, the more flexible alternative would probably be to do as you're suggesting (if I understand you correctly) -- let the function handle the parsing, with a base implementation and utilities provided. The class names would be mapped to function names in the config file. - Jon I'd just thought about listing class names in the config file. The functions would probably need to extend a subclass of ValueSource which will handle argument parsing for the function, so you won't need to hard code the parsing in a VSParser subclass. I think this might simplify the existing code a bit. You might have to do a bit of reflection to instantiate the function. Did you have an alternate approach in mind? Are there any other things this would need to do? Is anyone else working on this? Tom On 9/18/07, Jon Pierce [EMAIL PROTECTED] wrote: I see Yonik recently opened an issue in JIRA to track the addition of pluggable functions (https://issues.apache.org/jira/browse/SOLR-356). Any chance this will be implemented soon? It would save users like me from having to hack the Solr source or write custom request handlers for trivial additions (e.g., adding a distance function), not to mention changes to downstream dependencies (e.g., solr-ruby). Perhaps a reflection-based approach would do the trick? - Jon
Formula for open file descriptors
Hi-
In early June Mike Klass posted a formula for the number of file descriptors
needed by Solr:
For each segment, 7 + num indexed fields per segment.
There should be log_{base mergefactor}(numDocs) * mergeFactor
segments, approximately.
Is this still true?
Thanks,
Lance Norskog
Re: pluggable functions
On 9/18/07, Jon Pierce [EMAIL PROTECTED] wrote: Reflection could be used to look up and invoke the constructor with appropriately-typed arguments. If we assume only primitive types and ValueSources are used, I don't think it would be too hard to craft a drop-in replacement that works with existing implementations. In any case, the more flexible alternative would probably be to do as you're suggesting (if I understand you correctly) -- let the function handle the parsing, The parser is a quick hack I threw together, and any value source factories should not be exposed to it. It seems like either 1) a value source factory would expose the types it expects or 2) a value source factory would take a ListValueSource and throw a ParseException if it didn't get what it expected Reflection might be fine if the cost of construction via reflection ends up being small compared to the parsing itself. -Yonik
Re: Formula for open file descriptors
On 18-Sep-07, at 5:39 PM, Lance Norskog wrote:
Hi-
In early June Mike Klass posted a formula for the number of file
descriptors
needed by Solr:
For each segment, 7 + num indexed fields per segment.
There should be log_{base mergefactor}(numDocs) * mergeFactor
segments, approximately.
Is this still true?
No, it is a constant ~10 files per segment now (no per-field file
descriptor count).
Unfortunately, I forgot that a set of file descriptors is needed for
each Reader -- so multiply the total by 2 at least.
The formula's seeming mathematical nature shouldn't be confused for
precision: I wouldn't use it for anything other than an order-mag
level estimation of the number of fds needed.
-Mike
How can i make a distribute search on Solr?
Hi everyone, I successfully do the Collection Distribution on two Linux servers - one master with one slave and sync the index data. How can I make a search request to master server and receive the response by all slave servers? OR it should be manually controlled? Thanks Best Regards. Jarvis .
Re: pluggable functions
(NOTE: this discussion probably makes more sense on solr-dev. future replies should probably go there, or in SOLR-334.) : The parser is a quick hack I threw together, and any value source : factories should not be exposed to it. It seems like either : 1) a value source factory would expose the types it expects : or : 2) a value source factory would take a ListValueSource and throw a : ParseException if it didn't get what it expected : : Reflection might be fine if the cost of construction via reflection : ends up being small compared to the parsing itself. Another option is to assume that if people are writing their own ValueSoures and loading them into solr as plugins, they could write their FunctionParser subclass that knows about those ValueSoures and then register that FunctionParser -- the key being to make it easy to subclass a FunctionParser to add your own functions (without needing to cut/paste a tone of stuff like you do now) To me the key differentiator between something like this, and something like Tokenizer/TokenFilter factories is that with those, you want to be able to mix/match them at run time a lot -- but i'm guessing once you write a ValueSource and you want the function parser to use whenever it sees foo(...) that's not something you really need to change with each Solr install (or have one function parser for one request handler, and a different one for another reuqest handler) -Hoss
RE: How can i make a distribute search on So lr?
There are two federated/distributed search implementations that are still a few weeks away from maturity: https://issues.apache.org/jira/browse/SOLR-255https://issues.apache.org/jira/browse/SOLR-303Any help in testing them would definitely be appreciated. BUT, if you decide to roll your own, take a look at the following wiki page for details on the complexity of the task: http://wiki.apache.org/solr/FederatedSearch Good luck! Thanks, Stu -Original Message- From: ¹ý¼Ñ Sent: Wednesday, September 19, 2007 12:24am To: solr-user@lucene.apache.org Subject: How can i make a distribute search on Solr? Hi everyone, I successfully do the Collection Distribution on two Linux servers - one master with one slave and sync the index data. How can I make a search request to master server and receive the response by all slave servers? OR it should be manually controlled? Thanks Best Regards. Jarvis .
RE: How can i make a distribute search on Solr?
Helpful information. So it means that distributed search is not a basic component in Solr project. Thanks Best Regards. Jarvis . -Original Message- From: Stu Hood [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 19, 2007 12:55 PM To: solr-user@lucene.apache.org Subject: RE: How can i make a distribute search on Solr? There are two federated/distributed search implementations that are still a few weeks away from maturity: https://issues.apache.org/jira/browse/SOLR-255https://issues.apache.org/jira/browse/SOLR-303Any help in testing them would definitely be appreciated. BUT, if you decide to roll your own, take a look at the following wiki page for details on the complexity of the task: http://wiki.apache.org/solr/FederatedSearch Good luck! Thanks, Stu -Original Message- From: ¹ý¼Ñ Sent: Wednesday, September 19, 2007 12:24am To: solr-user@lucene.apache.org Subject: How can i make a distribute search on Solr? Hi everyone, I successfully do the Collection Distribution on two Linux servers - one master with one slave and sync the index data. How can I make a search request to master server and receive the response by all slave servers? OR it should be manually controlled? Thanks Best Regards. Jarvis .
Re: How can i make a distribute search on Solr?
So it means that distributed search is not a basic component in Solr project. I think you just need load balancing. Solr is not a load balancer, you need to find something that works for you and configure that elsewhere. Solr works fine without persistent connections, so simple round robin DNS but it works find. Depending on your usage/loads/requirements it may or may not make sense to have your master DB in the mix. Stu is referring to Federated Search - where each index has some of the data and results are combined before they are returned. This is not yet supported out of the box ryan
