Re: Can solr return documents which don't match a query?
Or add a new tag: the "NO TAG" at index time and search for that. (If you have the possibility to reindex at least that one, first, time.) Would clear things up for developers/admins looking at the stuff in some months... Chantal Yonik Seeley schrieb: "return all documents which either match query1 or don't match query 2" query1 (*:* -query2) -Yonik http://www.lucidimagination.com
Re: capturing field length into a stored document field
Here's a hybrid solution. Add a filter to the field in question that counts all the tokens and at the end outputs a token of the form __numtokens.__. This eliminates the need to retokenize the field again. Also, bucket the numbers, either by some factor of ten, or base 2, so that there aren't so many different token types produced. This has a space advantage over storing in a field, especially since the information isn't needed at query time anyway. mike.schultz wrote: > > For various statistics I collect from an index it's important for me to > know the length (measured in tokens) of a document field. I can get that > information to some degree from the "norms" for the field but a) the > resolution isn't that great, and b) more importantly, if boosts are used > it's almost impossible to get lengths from this. > > Here's two ideas I was thinking about that maybe some can comment on. > > 1) Use copyto to copy the field in question, fieldA to an addition field, > fieldALength, which has an extra filter that just counts the tokens and > only outputs a token representing the length of the field. This has the > disadvantage of retokenizing basically the whole document (because the > field in question is basically the body). Plus I would think littering > the term space with these tokens might be bad for performance, I'm not > sure. > > 2) Add a filter to the field in question which again counts the tokens. > This filter allows the regular tokens to be indexed as usual but somehow > manages to get the token-count into a stored field of the document. This > has the advantage of not having to retokenize the field and instead of > littering the token space, the count becomes docdata for each doc. Can > this be done? Maybe using threadLocal to temporarily store the count? > > Thanks. > > -- View this message in context: http://www.nabble.com/capturing-field-length-into-a-stored-document-field-tp25297690p25339584.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Can solr return documents which don't match a query?
> "return all documents which either match query1 or don't match query 2" query1 (*:* -query2) -Yonik http://www.lucidimagination.com
Can solr return documents which don't match a query?
Hi everyone, I have what's probably a loaded question about how Solr parses queries. Basically, the client wants to restructure the way they handle permissions. Instead of assigning specific permissions ids (as we've done up until this point), they want to structure permissions by adding a particular filter query depending on the products that the user has purchased. In principle, it could save a lot of reindexing time because when products change, it simply changes the filter queries, instead of requiring reindexing of large portions of the documents in the index. However, I'm hitting a snag... Basically, here is the way we'd like to structure the queries: Documents can have zero or more tags in a particular field assigned to them. All permissions are essentially tied to the tags in this field. For each product a user has purchased, there are a certain number of tags associated with it. Users should be able to see all documents which match any of these tags, as well as all documents which don't match *any* tag associated with a product. Basically, this means that users can see any documents which match their specific products, as well as any documents which aren't bound to any product in particular (documents which are essentially open to anyone). Let's say that the following tags represent all the restricted tags in the system: tag1 tag2 tag3 tag4 The current user is allowed access to documents with tags tag1 and tag2, and documents which have none of these tags. So, by my own logic, I'd make the query: ((my_field:tag1) OR (my_field:tag2) OR (-my_field:tag1 AND - my_field:tag2 AND -my_field:tag3 AND -my_field:tag4)) Keep in mind that "my_field" can have other tags beyond tags 1-4 which essentially have no impact on permissions. Obviously, I'm writing here because this query doesn't work... at all. I see according to the wikis that, apparently, - or "NOT" *excludes* all documents which have the tag in question. So this is just not the right syntax for what I'm doing. What I can't seem to find is a syntax which says "return all documents which don't match this query", or perhaps more importantly, "return all documents which either match query1 or don't match query 2". I tried using the "_query_" syntax but no luck there either. Am I mistaken, or is there simply no way in SOLR to return the documents which didn't match the specified query? If not, I fear there's very little potential for optimization here Reindexing all affected documents for a product every time a product's parameters change is becoming very cumbersome - it can take hours. We'd really like to find a way to avoid having to do that. If anyone can help figure out a way to actually do this I'd be very grateful. Thanks! -- Steve
abortOnConfigurationError=false not taking the effect in solr 1.3
Hi, We have set false in solrconfig.xml for all the cores but still whenever any configuration error occurs search does not work in any of the cores. Is there any other configuration we have to set so as to not affect search in the cores which are configured correctly? For one of the core, we got following error today but then none of the core is working after that. The below message suggests to add abortOnConfigurationError in solr.xml but we have it added in solrconfig.xml for all the cores. Do we need to add in solr.xml? If yes, where? Please suggest what is the right way to configure so that if one core fails due to configuration errors, all other cores remain unaffected? * Check your log files for more detailed information on what may be wrong. If you want solr to continue after configuration errors, change:false in solr.xml java.lang.RuntimeException: java.io.IOException: Cannot create directory: /usr/local/app/data/search/core09/index *** -- View this message in context: http://www.nabble.com/abortOnConfigurationError%3Dfalse-not-taking-the-effect-in-solr-1.3-tp25332254p25332254.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Netbeans and Solr : Whac-A-Mole
Mark Miller wrote:
> Setting Solr up to easily run tests through the IDE should be about as
> simple with any IDE of your choosing.
>
> I do it with eclipse. It takes a couple basic steps:
>
> 1. Add the correct libraries to your classpath - all the jars in lib and
> example/lib (for jetty jars).
>
> 2. Add the build/solr build/solrj build/test folders to your classpath.
>
> 3. Set your working directory to
> ${workspace_loc:solrtrunk}/src/test/test-files/ (In eclipse, the opening
> variable here is a path to the project base dir - for other IDE's,
> change to what makes sense).
>
>
> Now you should be good to go.
>
> (In eclipse, its best to add the jars/folders to the project classpath,
> rather than to the single test Run configuration - else you will have to
> do it for every test - instead, add it to the project build path and
> then set the working directory for each test Run configuration.)
>
> Now its easy to run tests one at a time, or to debug, or whatever. Not
> quite as simple to get going as Lucene, but once you have it up, its the
> same dev pleasure.
>
> - Mark
>
P.S. to extend to contribs, just add their jars and class folders as well.
--
- Mark
http://www.lucidimagination.com
Re: Netbeans and Solr : Whac-A-Mole
Setting Solr up to easily run tests through the IDE should be about as
simple with any IDE of your choosing.
I do it with eclipse. It takes a couple basic steps:
1. Add the correct libraries to your classpath - all the jars in lib and
example/lib (for jetty jars).
2. Add the build/solr build/solrj build/test folders to your classpath.
3. Set your working directory to
${workspace_loc:solrtrunk}/src/test/test-files/ (In eclipse, the opening
variable here is a path to the project base dir - for other IDE's,
change to what makes sense).
Now you should be good to go.
(In eclipse, its best to add the jars/folders to the project classpath,
rather than to the single test Run configuration - else you will have to
do it for every test - instead, add it to the project build path and
then set the working directory for each test Run configuration.)
Now its easy to run tests one at a time, or to debug, or whatever. Not
quite as simple to get going as Lucene, but once you have it up, its the
same dev pleasure.
- Mark
Re: Netbeans and Solr : Whac-A-Mole
Shalin, do you think you can share more details about your IDEA setup for Solr testing? I am still having some issues with some tests (not all tests can find solr configuration files). Best regards, Lukas On Mon, Sep 7, 2009 at 2:35 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Mon, Sep 7, 2009 at 5:58 PM, Fergus McMenemie > wrote: > > > >This testcase is quite independent of anything in Solr. It is a > > >standalone utility and the only dependency is stax. > > >discalimer (I run these testcases from Intellij and command line) > > >BTW are you using XpathRecordReader outside of DIH? > > > > Nobel, > > > > Is there a better way to test and play with XPathRecordReader.java > > other than > > > > ant -Dtestcase=TestXPathRecordReader test > > > > Which takes 8secs to run here? I am not using XpathRecordReader > > outside of DIH, but looking to see how I would add support for > > xpaths such as //a. > > > > > The target takes a lot of time because it has to go through all the > test-cases in core and contribs trying to match the value given in > -Dtestcase. > > You could also do ant -Dtestcase=TestXPathRecordReader test-contrib which > should be a little faster. I run individual test cases directly through > IDEA > which avoids these extra steps. > > -- > Regards, > Shalin Shekhar Mangar. >
Re: Netbeans and Solr : Whac-A-Mole
It is possible to use XPathRecordReader directly with a couple of lines of code as demonstrated in the JUnit testcases. just add the stax jars in the classpath On Mon, Sep 7, 2009 at 5:58 PM, Fergus McMenemie wrote: >>This testcase is quite independent of anything in Solr. It is a >>standalone utility and the only dependency is stax. >>discalimer (I run these testcases from Intellij and command line) >>BTW are you using XpathRecordReader outside of DIH? > > Nobel, > > Is there a better way to test and play with XPathRecordReader.java > other than > > ant -Dtestcase=TestXPathRecordReader test > > Which takes 8secs to run here? I am not using XpathRecordReader > outside of DIH, but looking to see how I would add support for > xpaths such as //a. > > Fergus. > >> >>On Mon, Sep 7, 2009 at 3:26 PM, Fergus McMenemie wrote: >>> Hello all, >>> >>> I would appreciate help from somebody who has set up Solr within >>> netbeans, I am wanting to do more work with DIH and particularly its >>> XpathEntityProcessor stuff. I wish to preform the following from >>> within the IDE >>> >>> ant -Dtestcase=TestXPathRecordReader.java test >>> >>> I have spent a few hours playing Whac-A-Mole with classpath and source >>> settings. In the end I got it down to zero flags, but I then added >>> some test cases and the scanner thing then went off and flagged dozens >>> files with undefined classes I removed my change but the rescan did not >>> remove the dozens of flagged files. >>> >>> PS: I am a total netbeans newbie. >>> >>> -- >>> >>> === >>> Fergus McMenemie Email:fer...@twig.me.uk >>> Techmore Ltd Phone:(UK) 07721 376021 >>> >>> Unix/Mac/Intranets Analyst Programmer >>> === >>> >> >> >> >>-- >>- >>Noble Paul | Principal Engineer| AOL | http://aol.com > > -- > > === > Fergus McMenemie Email:fer...@twig.me.uk > Techmore Ltd Phone:(UK) 07721 376021 > > Unix/Mac/Intranets Analyst Programmer > === > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Faceting optimization
If you add &debugQuery=true to your query, it should give you a breakdown of time spent in the various components. Also, are you doing any warming? You might also look at the new faceting method in Solr 1.4. On Sep 7, 2009, at 6:37 AM, Sébastien Lamy wrote: Hi I'm currently trying to optimize the response time of my solr server. I found one aberration and hope you may be able to help me solve it: If, considering the whole document index, there is a lot of possible values for a field, asking for facet on that field dramatically increase response time. Even if the search returns only one document, with only one facet value for that field. This is shown by the three requests at the bottom of this mail. It seems to me that solr looks at all the possible values in the whole index for the faceted field. Whereas it should look at the possible values only for the documents in the results, wich would be a lot faster. Is there a way asking him to do so? --- Let's look at this three requests: 1- This request returns only one document and take 3ms http://localhost:8983/solr/select/? rows=10& q=(available_owner_display_name_s_facet:%22mag%22)+AND+type_s:[T0+TO +T9] 2- This request returns one document, and its facets for one field. It takes about 1000ms. The facet on a_10_alpha_sort returns only one value: "air du temps". But overall the whole index, there is a lot of values (>10 000) for a_10_alpha_sort. http://localhost:8983/solr/select/? facet=true& rows=10& q=(available_owner_display_name_s_facet:%22mag%22)+AND+type_s:[T0+TO +T9]& facet.field=a_10_alpha_sort& f.a_10_alpha_sort.facet.mincount=1& f.a_10_alpha_sort.facet.sort=true& f.a_10_alpha_sort.facet.limit=8& 3- This request includes the value "air du temps" in the search string. It takes 3ms http://localhost:8983/solr/select/? rows=10& q=(available_owner_display_name_s_facet:%22mag%22+AND +a_10_alpha_sort:"air+du+temps")+AND+type_s:[T0+TO+T9] Here is the description of the faceted field in my schema: this is a single-valued field, with no tokens. indexed="true" stored="false" multivalued="false"/> sortMissingLast="true" omitNorms="true"> -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: Netbeans and Solr : Whac-A-Mole
>On Mon, Sep 7, 2009 at 5:58 PM, Fergus McMenemie wrote: > >> >This testcase is quite independent of anything in Solr. It is a >> >standalone utility and the only dependency is stax. >> >discalimer (I run these testcases from Intellij and command line) >> >BTW are you using XpathRecordReader outside of DIH? >> >> Nobel, >> >> Is there a better way to test and play with XPathRecordReader.java >> other than >> >> ant -Dtestcase=TestXPathRecordReader test >> >> Which takes 8secs to run here? I am not using XpathRecordReader >> outside of DIH, but looking to see how I would add support for >> xpaths such as //a. >> >> >The target takes a lot of time because it has to go through all the >test-cases in core and contribs trying to match the value given in >-Dtestcase. > >You could also do ant -Dtestcase=TestXPathRecordReader test-contrib which >should be a little faster. I run individual test cases directly through IDEA >which avoids these extra steps. > Shalin, Hmm, 6 seconds. I looked up IDEA and I guess I should be able to use it for free while working on solr. Is it easier to setup and come up the learning curve? Regards Fergus. -- === Fergus McMenemie Email:fer...@twig.me.uk Techmore Ltd Phone:(UK) 07721 376021 Unix/Mac/Intranets Analyst Programmer ===
Re: Netbeans and Solr : Whac-A-Mole
On Mon, Sep 7, 2009 at 5:58 PM, Fergus McMenemie wrote: > >This testcase is quite independent of anything in Solr. It is a > >standalone utility and the only dependency is stax. > >discalimer (I run these testcases from Intellij and command line) > >BTW are you using XpathRecordReader outside of DIH? > > Nobel, > > Is there a better way to test and play with XPathRecordReader.java > other than > > ant -Dtestcase=TestXPathRecordReader test > > Which takes 8secs to run here? I am not using XpathRecordReader > outside of DIH, but looking to see how I would add support for > xpaths such as //a. > > The target takes a lot of time because it has to go through all the test-cases in core and contribs trying to match the value given in -Dtestcase. You could also do ant -Dtestcase=TestXPathRecordReader test-contrib which should be a little faster. I run individual test cases directly through IDEA which avoids these extra steps. -- Regards, Shalin Shekhar Mangar.
Re: Netbeans and Solr : Whac-A-Mole
>This testcase is quite independent of anything in Solr. It is a >standalone utility and the only dependency is stax. >discalimer (I run these testcases from Intellij and command line) >BTW are you using XpathRecordReader outside of DIH? Nobel, Is there a better way to test and play with XPathRecordReader.java other than ant -Dtestcase=TestXPathRecordReader test Which takes 8secs to run here? I am not using XpathRecordReader outside of DIH, but looking to see how I would add support for xpaths such as //a. Fergus. > >On Mon, Sep 7, 2009 at 3:26 PM, Fergus McMenemie wrote: >> Hello all, >> >> I would appreciate help from somebody who has set up Solr within >> netbeans, I am wanting to do more work with DIH and particularly its >> XpathEntityProcessor stuff. I wish to preform the following from >> within the IDE >> >> ant -Dtestcase=TestXPathRecordReader.java test >> >> I have spent a few hours playing Whac-A-Mole with classpath and source >> settings. In the end I got it down to zero flags, but I then added >> some test cases and the scanner thing then went off and flagged dozens >> files with undefined classes I removed my change but the rescan did not >> remove the dozens of flagged files. >> >> PS: I am a total netbeans newbie. >> >> -- >> >> === >> Fergus McMenemie Email:fer...@twig.me.uk >> Techmore Ltd Phone:(UK) 07721 376021 >> >> Unix/Mac/Intranets Analyst Programmer >> === >> > > > >-- >- >Noble Paul | Principal Engineer| AOL | http://aol.com -- === Fergus McMenemie Email:fer...@twig.me.uk Techmore Ltd Phone:(UK) 07721 376021 Unix/Mac/Intranets Analyst Programmer ===
Re: Netbeans and Solr : Whac-A-Mole
We use command-line for most stuff except editing/debugging! 2009/9/7 Noble Paul നോബിള് नोब्ळ् > This testcase is quite independent of anything in Solr. It is a > standalone utility and the only dependency is stax. > discalimer (I run these testcases from Intellij and command line) > BTW are you using XpathRecordReader outside of DIH? > > On Mon, Sep 7, 2009 at 3:26 PM, Fergus McMenemie wrote: > > Hello all, > > > > I would appreciate help from somebody who has set up Solr within > > netbeans, I am wanting to do more work with DIH and particularly its > > XpathEntityProcessor stuff. I wish to preform the following from > > within the IDE > > > > ant -Dtestcase=TestXPathRecordReader.java test > > > > I have spent a few hours playing Whac-A-Mole with classpath and source > > settings. In the end I got it down to zero flags, but I then added > > some test cases and the scanner thing then went off and flagged dozens > > files with undefined classes I removed my change but the rescan did not > > remove the dozens of flagged files. > > > > PS: I am a total netbeans newbie. > > > > -- > > > > === > > Fergus McMenemie > > Email:fer...@twig.me.uk > > Techmore Ltd Phone:(UK) 07721 376021 > > > > Unix/Mac/Intranets Analyst Programmer > > === > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com >
Re: Netbeans and Solr : Whac-A-Mole
This testcase is quite independent of anything in Solr. It is a standalone utility and the only dependency is stax. discalimer (I run these testcases from Intellij and command line) BTW are you using XpathRecordReader outside of DIH? On Mon, Sep 7, 2009 at 3:26 PM, Fergus McMenemie wrote: > Hello all, > > I would appreciate help from somebody who has set up Solr within > netbeans, I am wanting to do more work with DIH and particularly its > XpathEntityProcessor stuff. I wish to preform the following from > within the IDE > > ant -Dtestcase=TestXPathRecordReader.java test > > I have spent a few hours playing Whac-A-Mole with classpath and source > settings. In the end I got it down to zero flags, but I then added > some test cases and the scanner thing then went off and flagged dozens > files with undefined classes I removed my change but the rescan did not > remove the dozens of flagged files. > > PS: I am a total netbeans newbie. > > -- > > === > Fergus McMenemie Email:fer...@twig.me.uk > Techmore Ltd Phone:(UK) 07721 376021 > > Unix/Mac/Intranets Analyst Programmer > === > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Exact Word Search
Hi Shalin, My search is based on the following fields in schema.xml Let me know if you need anything else? Regards Bhaskar --- On Fri, 9/4/09, Shalin Shekhar Mangar wrote: From: Shalin Shekhar Mangar Subject: Re: Exact Word Search To: solr-user@lucene.apache.org Date: Friday, September 4, 2009, 5:51 AM On Fri, Sep 4, 2009 at 6:06 PM, bhaskar chandrasekar wrote: > > Hi, > > > I have integrated Solr with Carrot2 Cluster Engine (v 3.1.0). > > Carrot2 is used as a presentation layer. Carrot2 sends requested query to > external source (Solr) and get results from Solr. > Carrot2 may not be responsible for forming Query. It would have been > handled from Solr end. > Can you post the exact query that your application or Carrot2 is sending to Solr? Can you also list the Solr field and type defined in schema.xml which is being searched? > > Please help me with the below scenarios. > > Scenario: (Please DO NOT consider any case sensitive) > > Assuming I give "bhaskar" as input string > It should give me search results pertaining to word ‘bhaskar’ only. > > I am expecting output like below database query > Select * from MASTER where name =’bhaskar’; > > Above query suppose to return matched records for ‘bhaskar’.. > Use a solr.TextField with KeywordTokenizer and LowerCaseFilter and search with q=field-name:"field-value" -- Regards, Shalin Shekhar Mangar.
Faceting optimization
Hi I'm currently trying to optimize the response time of my solr server. I found one aberration and hope you may be able to help me solve it: If, considering the whole document index, there is a lot of possible values for a field, asking for facet on that field dramatically increase response time. Even if the search returns only one document, with only one facet value for that field. This is shown by the three requests at the bottom of this mail. It seems to me that solr looks at all the possible values in the whole index for the faceted field. Whereas it should look at the possible values only for the documents in the results, wich would be a lot faster. Is there a way asking him to do so? --- Let's look at this three requests: 1- This request returns only one document and take 3ms http://localhost:8983/solr/select/? rows=10& q=(available_owner_display_name_s_facet:%22mag%22)+AND+type_s:[T0+TO+T9] 2- This request returns one document, and its facets for one field. It takes about 1000ms. The facet on a_10_alpha_sort returns only one value: "air du temps". But overall the whole index, there is a lot of values (>10 000) for a_10_alpha_sort. http://localhost:8983/solr/select/? facet=true& rows=10& q=(available_owner_display_name_s_facet:%22mag%22)+AND+type_s:[T0+TO+T9]& facet.field=a_10_alpha_sort& f.a_10_alpha_sort.facet.mincount=1& f.a_10_alpha_sort.facet.sort=true& f.a_10_alpha_sort.facet.limit=8& 3- This request includes the value "air du temps" in the search string. It takes 3ms http://localhost:8983/solr/select/? rows=10& q=(available_owner_display_name_s_facet:%22mag%22+AND+a_10_alpha_sort:"air+du+temps")+AND+type_s:[T0+TO+T9] Here is the description of the faceted field in my schema: this is a single-valued field, with no tokens. stored="false" multivalued="false"/> sortMissingLast="true" omitNorms="true">
Re: Netbeans and Solr : Whac-A-Mole
>We've set-up NetBeans with Solr but we are using command-line for most of >the stuff except for editing the code. > >Does your code build from NetBeans? If not what errors do you see? The code builds and runs from net beans because the underlying build.xml file is being used. But when you want to run testcases... you are doing that from the command line? Are you are only using the IDE as an editor? >Regards >Rajan > >On Mon, Sep 7, 2009 at 3:26 PM, Fergus McMenemie wrote: > >> Hello all, >> >> I would appreciate help from somebody who has set up Solr within >> netbeans, I am wanting to do more work with DIH and particularly its >> XpathEntityProcessor stuff. I wish to preform the following from >> within the IDE >> >> ant -Dtestcase=TestXPathRecordReader.java test >> >> I have spent a few hours playing Whac-A-Mole with classpath and source >> settings. In the end I got it down to zero flags, but I then added >> some test cases and the scanner thing then went off and flagged dozens >> files with undefined classes I removed my change but the rescan did not >> remove the dozens of flagged files. >> >> PS: I am a total netbeans newbie. >> >> -- >> >> === >> Fergus McMenemie >> Email:fer...@twig.me.uk >> Techmore Ltd Phone:(UK) 07721 376021 >> >> Unix/Mac/Intranets Analyst Programmer >> === >> -- === Fergus McMenemie Email:fer...@twig.me.uk Techmore Ltd Phone:(UK) 07721 376021 Unix/Mac/Intranets Analyst Programmer ===
Re: Netbeans and Solr : Whac-A-Mole
We've set-up NetBeans with Solr but we are using command-line for most of the stuff except for editing the code. Does your code build from NetBeans? If not what errors do you see? Regards Rajan On Mon, Sep 7, 2009 at 3:26 PM, Fergus McMenemie wrote: > Hello all, > > I would appreciate help from somebody who has set up Solr within > netbeans, I am wanting to do more work with DIH and particularly its > XpathEntityProcessor stuff. I wish to preform the following from > within the IDE > > ant -Dtestcase=TestXPathRecordReader.java test > > I have spent a few hours playing Whac-A-Mole with classpath and source > settings. In the end I got it down to zero flags, but I then added > some test cases and the scanner thing then went off and flagged dozens > files with undefined classes I removed my change but the rescan did not > remove the dozens of flagged files. > > PS: I am a total netbeans newbie. > > -- > > === > Fergus McMenemie > Email:fer...@twig.me.uk > Techmore Ltd Phone:(UK) 07721 376021 > > Unix/Mac/Intranets Analyst Programmer > === >
Netbeans and Solr : Whac-A-Mole
Hello all, I would appreciate help from somebody who has set up Solr within netbeans, I am wanting to do more work with DIH and particularly its XpathEntityProcessor stuff. I wish to preform the following from within the IDE ant -Dtestcase=TestXPathRecordReader.java test I have spent a few hours playing Whac-A-Mole with classpath and source settings. In the end I got it down to zero flags, but I then added some test cases and the scanner thing then went off and flagged dozens files with undefined classes I removed my change but the rescan did not remove the dozens of flagged files. PS: I am a total netbeans newbie. -- === Fergus McMenemie Email:fer...@twig.me.uk Techmore Ltd Phone:(UK) 07721 376021 Unix/Mac/Intranets Analyst Programmer ===
Re: Field Collapsing (was Re: Schema for group/child entity setup)
Great. Nice site and very similar to my requirements. thanks. So, right now, you get all field values by default? Right now, no field values are returned for the collapsed documents. The patch which will be committed soon will add this functionality. R. Tan wrote: Great. Nice site and very similar to my requirements. There's work on the patch that is being done now which will enable you to ask for specific field values of the collapsed documents using a dedicated request parameter. So, right now, you get all field values by default? On Sun, Sep 6, 2009 at 3:58 AM, Uri Boness wrote: You can check out http://www.ilocal.nl. If you search for a bank in Amsterdam then you'll see that a lot of the results are collapsed. For this we used an older version of this patch (which works on 1.3) but a lot has changed since then. We're currently using this patch on another project, but it's not live yet. Uri R. Tan wrote: Thanks Uri. Your personal suggestion is appreciated and I think I'll follow your advice. We're still early in development and 1.4 would be a good choice. I hope I can get field collapsing to work with my requirements. Do you know any live site using field collapsing already? On Sat, Sep 5, 2009 at 5:57 PM, Uri Boness wrote: There's work on the patch that is being done now which will enable you to ask for specific field values of the collapsed documents using a dedicated request parameter. This work is not committed yet to the latest patch, but will be very soon. There is of course a drawback to that as well, the collapsed documents set can be very large (depends on your data of course) in which case the returned result which includes the fields values can be rather large, which will impact performance, this is why this feature will be enabled only if you specify this extra parameter - by default no field values will be returned. AFAIK, the latest patch should work fine with the latest build. Martijn (which is the main maintainer of this patch) tries to keep it up to date with the latest builds. But I guess the safest way is to work with the nightly build of the same date as the latest patch (though I would give it a try first with the latest build). BTW, it's not an official suggestion from the Solr development team, but if you ask me, if you have to choose now whether to use 1.3 or 1.4-dev, I would go for the later. 1.4 is supposed to be released in the upcoming week or two and it bring loads of bug fixes, enhancements and extra functionality. But again, this is my personal suggestion. cheers, Uri R. Tan wrote: Okay. Thanks for giving an insight on how it works in general. Without trying it myself, are the field values for the collapsed ones also part of the results data? What is the latest build that is safe to use on a production environment? I'd probably go for that and use field collapsing. Thank you very much. On Fri, Sep 4, 2009 at 4:49 AM, Uri Boness wrote: The collapsed documents are represented by one "master" document which can be part of the normal search result (the doc list), so pagination just works as expected, meaning taking only the returned documents in account (ignoring the collapsed ones). As for the scoring, the "master" document is actually the document with the highest score in the collapsed group. As for Solr 1.3 compatibility... well... it's very hart to tell. All latest patch are certainly *not* 1.3 compatible (I think they're also depending on some changes in lucene which are not available for solr 1.3). I guess you'll have to try some of the old patches, but I'm not sure about their stability. cheers, Uri R. Tan wrote: Thanks Uri. How does paging and scoring work when using field collapsing? What patch works with 1.3? Is it production ready? R On Thu, Sep 3, 2009 at 3:54 PM, Uri Boness wrote: The development on this patch is quite active. It works well for single solr instance, but distributed search (ie. shards) is not yet supported. Using this page you can group search results based on a specific field. There are two flavors of field collapsing - adjacent and non-adjacent, the former collapses only document which happen to be located next to each other in the otherwise-non-collapsed results set. The later (the non-adjacent) one collapses all documents with the same field value (regardless of their position in the otherwise-non-collapsed results set). Note, that non-adjacent performs better than adjacent one. There's currently discussion to extend this support so in addition to collapsing the documents, extra information will be returned for the collapsed documents (see the discussion on the issue page). Uri R. Tan wrote: I think this is what I'm looking for. What is the status of this patch? On Thu, Sep 3, 2009 at 12:00 PM, R. Tan wrote: Hi Solrers, I would like to get your o
