Re: change data dir location
that is fine. you can use any variable in solrconfig.xml as ${the.solr.data.dir}} set the value of the.solr.data.dir as -Dthe.solr.data.dir=%SOME_WINDOWS_PROPERTY% On Fri, Jun 12, 2009 at 11:16 AM, Ashish P wrote: > > I don't want to set in solrConfig.xml. I want solr to take from my config > file or from system property. > Thanks, > Ashish > > > Noble Paul നോബിള് नोब्ळ्-2 wrote: >> >> set the value in solrconfig.xml to what you like >> >> On Fri, Jun 12, 2009 at 10:38 AM, Ashish P >> wrote: >>> >>> I want to change data dir location and not use solrConfig.xml. >>> So on windows I removed dataDir xonfiguration from solrConfig.xml and >>> tried >>> setting 'solr.data.dir' system property to my location. >>> But it looks like solr is not picking up this property on windows. >>> Anybody experienced the same??? >>> Please share... >>> Thanks, >>> Ashish >>> -- >>> View this message in context: >>> http://www.nabble.com/change-data-dir-location-tp23992946p23992946.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> >> >> -- >> - >> Noble Paul | Principal Engineer| AOL | http://aol.com >> >> > > -- > View this message in context: > http://www.nabble.com/change-data-dir-location-tp23992946p23993210.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Getting details from
I guess not. u will have to make a query to get the no:of docs matching the query probably this can be an enhancement to Solr On Thu, Jun 11, 2009 at 8:16 PM, Jacob Elder wrote: > Hello, > > Is there any way to get the number of deleted records from a delete request? > > I'm sending: > > type_i:(2 OR 3) AND creation_time_rl:[0 TO > 124426080] > > And getting: > > > > 0 name="QTime">2 > > > This is Solr 1.3. > > -- > Jacob Elder > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Faceting on text fields
Thanks Otis! Do you know under what circumstances or application should we cluster the whole corpus of documents vs just the search results? Jeffrey On Fri, Jun 12, 2009 at 1:39 PM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > > Jeffrey, > > Are you looking to cluster a whole corpus of documents of just the search > results? If it's the latter, use Carrot2. If it's the former, look at > Mahout. Clustering top 1M matching documents doesn't really make sense. > Usually top 100-200 is sufficient. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message > > From: Jeffrey Tiong > > To: solr-user@lucene.apache.org > > Sent: Friday, June 12, 2009 12:44:55 AM > > Subject: Re: Faceting on text fields > > > > Hi all, > > > > We are thinking of using the carrot clustering too. But we saw that > carrot > > maybe can only cluster up to 1000 search snippets. Does anyone know how > can > > we cluster snippets that is much more than that ? (maybe in the million > > range?) > > > > And what is the difference between mahout and carrot? > > > > Thank! > > > > Jeffrey > > > > On Thu, Jun 11, 2009 at 9:47 PM, Michael Ludwig wrote: > > > > > Yao Ge schrieb: > > > > > >> BTW, Carrot2 has a very impressive Clustering Workbench (based on > > >> eclipse) that has built-in integration with Solr. If you have a Solr > > >> service running, it is a just a matter of point the workbench to it. > > >> The clustering results and visualization are amazing. > > >> (http://project.carrot2.org/download.html). > > >> > > > > > > A new world opens up for me ... > > > > > > Thanks for pointing out how cool this is! > > > > > > Hint for other newcomers: Open the View Menu to configure the details > of > > > how you perform your search, e.g. your Solr URL in case it differs from > > > the default, or your "summary field", which is what gets used to > analyze > > > the data in order to determine clusters, if I understand correctly. > > > > > > Michael Ludwig > > > > >
Re: change data dir location
I don't want to set in solrConfig.xml. I want solr to take from my config file or from system property. Thanks, Ashish Noble Paul നോബിള് नोब्ळ्-2 wrote: > > set the value in solrconfig.xml to what you like > > On Fri, Jun 12, 2009 at 10:38 AM, Ashish P > wrote: >> >> I want to change data dir location and not use solrConfig.xml. >> So on windows I removed dataDir xonfiguration from solrConfig.xml and >> tried >> setting 'solr.data.dir' system property to my location. >> But it looks like solr is not picking up this property on windows. >> Anybody experienced the same??? >> Please share... >> Thanks, >> Ashish >> -- >> View this message in context: >> http://www.nabble.com/change-data-dir-location-tp23992946p23992946.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com > > -- View this message in context: http://www.nabble.com/change-data-dir-location-tp23992946p23993210.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: change data dir location
set the value in solrconfig.xml to what you like On Fri, Jun 12, 2009 at 10:38 AM, Ashish P wrote: > > I want to change data dir location and not use solrConfig.xml. > So on windows I removed dataDir xonfiguration from solrConfig.xml and tried > setting 'solr.data.dir' system property to my location. > But it looks like solr is not picking up this property on windows. > Anybody experienced the same??? > Please share... > Thanks, > Ashish > -- > View this message in context: > http://www.nabble.com/change-data-dir-location-tp23992946p23992946.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Faceting on text fields
Jeffrey, Are you looking to cluster a whole corpus of documents of just the search results? If it's the latter, use Carrot2. If it's the former, look at Mahout. Clustering top 1M matching documents doesn't really make sense. Usually top 100-200 is sufficient. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Jeffrey Tiong > To: solr-user@lucene.apache.org > Sent: Friday, June 12, 2009 12:44:55 AM > Subject: Re: Faceting on text fields > > Hi all, > > We are thinking of using the carrot clustering too. But we saw that carrot > maybe can only cluster up to 1000 search snippets. Does anyone know how can > we cluster snippets that is much more than that ? (maybe in the million > range?) > > And what is the difference between mahout and carrot? > > Thank! > > Jeffrey > > On Thu, Jun 11, 2009 at 9:47 PM, Michael Ludwig wrote: > > > Yao Ge schrieb: > > > >> BTW, Carrot2 has a very impressive Clustering Workbench (based on > >> eclipse) that has built-in integration with Solr. If you have a Solr > >> service running, it is a just a matter of point the workbench to it. > >> The clustering results and visualization are amazing. > >> (http://project.carrot2.org/download.html). > >> > > > > A new world opens up for me ... > > > > Thanks for pointing out how cool this is! > > > > Hint for other newcomers: Open the View Menu to configure the details of > > how you perform your search, e.g. your Solr URL in case it differs from > > the default, or your "summary field", which is what gets used to analyze > > the data in order to determine clusters, if I understand correctly. > > > > Michael Ludwig > >
change data dir location
I want to change data dir location and not use solrConfig.xml. So on windows I removed dataDir xonfiguration from solrConfig.xml and tried setting 'solr.data.dir' system property to my location. But it looks like solr is not picking up this property on windows. Anybody experienced the same??? Please share... Thanks, Ashish -- View this message in context: http://www.nabble.com/change-data-dir-location-tp23992946p23992946.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceting on text fields
Hi all, We are thinking of using the carrot clustering too. But we saw that carrot maybe can only cluster up to 1000 search snippets. Does anyone know how can we cluster snippets that is much more than that ? (maybe in the million range?) And what is the difference between mahout and carrot? Thank! Jeffrey On Thu, Jun 11, 2009 at 9:47 PM, Michael Ludwig wrote: > Yao Ge schrieb: > >> BTW, Carrot2 has a very impressive Clustering Workbench (based on >> eclipse) that has built-in integration with Solr. If you have a Solr >> service running, it is a just a matter of point the workbench to it. >> The clustering results and visualization are amazing. >> (http://project.carrot2.org/download.html). >> > > A new world opens up for me ... > > Thanks for pointing out how cool this is! > > Hint for other newcomers: Open the View Menu to configure the details of > how you perform your search, e.g. your Solr URL in case it differs from > the default, or your "summary field", which is what gets used to analyze > the data in order to determine clusters, if I understand correctly. > > Michael Ludwig >
can Trie fields be stored?
Looking at the new examples of solr.TrieField http://svn.apache.org/repos/asf/lucene/solr/trunk/example/solr/conf/schema.xml I see that all have indexed="true" stored="false" in the field tpye definition. Does this mean that yo cannot ever store a value for one of these fields? I.e. if I want to do a range query and also return the values, I need to store the values in a separate field? Thanks, Peter -- Peter M. Wolanin, Ph.D. Momentum Specialist, Acquia. Inc. peter.wola...@acquia.com
Re: Query Filter fq with OR operator
Adding my vote for this feature. Another possible way to do this would be to introduce the "ofq" parameter .. so all of the fq's and AND'ed and the ofq's would be OR'd I've got some really complicated fq's that aren't going to cache well, introducing OR'd filters would allow me to use simpler filters and I expect performance would be a lot better Thanks -- View this message in context: http://www.nabble.com/Query-Filter-fq-with-OR-operator-tp23895837p23986336.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to get to highlitghting results using solrJ
the query contained some experimenting code. The correct one is: http://localhost:8983/solr/autocompleteCore/select?fl=name_display,importance,score&wt=xml&q={!boost b=log(importance)}(prefix1:"or" OR prefix2:"or")&hl=true&hl.fl=prefix1 Moreover, Is there a way to simply add the result of highlighting to the fl-parameter, so I can just read the annotated name (including highlighting) instead of the normal name. (analogously as you can apply 'score' to fl.) To me, this would seem like the perfect way to get the highlighted result without having to supply additional code in a client. You would only need to refer to the annotated field name... Britske wrote: > > first time I'm using highlighting and results work ok. > Im using it for an auto-suggest function. For reference I used the > following query: > > http://localhost:8983/solr/autocompleteCore/select?fl=name_display,importance,score,hl&id&wt=xml&q={!boost > b=log(importance)}(prefix1:"or" OR prefix2:"or")&hl=true&hl.fl=prefix1 > > However, when using solrJ I can't get to the actual highlighted results, > i.e: > > QueryResponse.getHighlighting() shows me a map as follows: > {2-1-57010={}, 2-7-8481={}, } which I can't use because the result is > empty.(?) > > but debugging I see a field: > QueryResponse._highlightingInfo with contents: > {1-4167147={prefix1=[Orlando Verenigde Staten]},} > which is exactly what I need. > > However there is no (public) method: > QueryRepsonse.getHighlightingInfo() ! > > what am I missing? > > thanks, > Britske > -- View this message in context: http://www.nabble.com/how-to-get-to-highlitghting-results-using-solrJ-tp23986063p23986127.html Sent from the Solr - User mailing list archive at Nabble.com.
how to get to highlitghting results using solrJ
first time I'm using highlighting and results work ok. Im using it for an auto-suggest function. For reference I used the following query: http://localhost:8983/solr/autocompleteCore/select?fl=name_display,importance,score,hl&id&wt=xml&q={!boost b=log(importance)}(prefix1:"or" OR prefix2:"or")&hl=true&hl.fl=prefix1 However, when using solrJ I can't get to the actual highlighted results, i.e: QueryResponse.getHighlighting() shows me a map as follows: {2-1-57010={}, 2-7-8481={}, } which I can't use because the result is empty.(?) but debugging I see a field: QueryResponse._highlightingInfo with contents: {1-4167147={prefix1=[Orlando Verenigde Staten]},} which is exactly what I need. However there is no (public) method: QueryRepsonse.getHighlightingInfo() ! what am I missing? thanks, Britske -- View this message in context: http://www.nabble.com/how-to-get-to-highlitghting-results-using-solrJ-tp23986063p23986063.html Sent from the Solr - User mailing list archive at Nabble.com.
localSolr + collapse
Hi, I am using localSolr with collapse. Query response gives two result sets one generated by localSolr based on geo distance and other by collapsing fields. How can I get combined result set? SolrConfig.xml explicit 0.01 true true true 1 collapse localsolr facet debug I am using solr 1.4 nightly build with collapse patch. Localsolr version is 1.5 Query: http://localhost:8080/solr/select?q=TV&qt=geo&collapse=true&collapse.fie ld=resource_id&lat=41.883784&long=-87.637668&radius=100 Regards Chandra
RE: Upgrading 1.2.0 to 1.3.0 solr
DO you have experience to upgrade from 1.2.0 to 1.3.0? In other words, do you have any suggestions or best if you have any docs or instructions for doing this. I appreciate if you can help me. Thanks Francis -Original Message- From: Ryan Grange [mailto:rgra...@dollardays.com] Sent: Thursday, June 11, 2009 8:39 AM To: solr-user@lucene.apache.org Subject: Re: Upgrading 1.2.0 to 1.3.0 solr I disagree with waiting that month. At this point, most of the kinks in the upgrade from 1.2 to 1.3 have been worked out. Waiting for 1.4 to come out risks you becoming a guinea pig for the upgrade procedure. Plus, if any show-stoppers come along delaying 1.4, you delay implementation of your auto-complete function. When 1.4 comes out, if it has any features you feel compel an upgrade, you can begin another round of testing and migration, but don't upgrade a production system just for the sake of being bleeding edge. Ryan T. Grange, IT Manager DollarDays International, Inc. rgra...@dollardays.com (480)922-8155 x106 Otis Gospodnetic wrote: > Francis, > > If you can wait another month or so, you could skip 1.3.0, and jump to 1.4 > which will be released soon. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > >> From: Francis Yakin >> To: "solr-user@lucene.apache.org" >> Sent: Wednesday, June 10, 2009 1:17:25 AM >> Subject: Upgrading 1.2.0 to 1.3.0 solr >> >> >> I am in process to upgrade our solr 1.2.0 to solr 1.3.0 >> >> Our solr 1.2.0 now is working fine, we just want to upgrade it cause we have >> an application that requires some function from 1.3.0( we call it >> autocomplete). >> >> Currently our config files on 1.2.0 are as follow: >> >> Solrconfig.xml >> Schema.xml ( we wrote this in house) >> Index_synonyms.txt ( we also modified and wrote this in house) >> Scripts.conf Protwords.txt Stopwords.txt Synonyms.txt >> >> I understand on 1.3.0 , it has new solrconfig.xml . >> >> My questions are: >> >> 1) what config files that I can reuse from 1.2.0 for 1.3.0 >> can I use the same schema.xml >> 2) Solrconfig.xml, can I use the 1.2.0 version or I have to stick with 1.3.0 >> If I need to stick with 1.3.0, what that I need to change. >> >> As of right I am testing it in my sandbox, so it doesn't work. >> >> Please advice, if you have any docs for upgrading 1.2.0 to 1.3.0 let me know. >> >> Thanks in advance >> >> Francis >> >> Note: I attached my solrconfigand schema.xml in this email >> >> >> >> -Inline Attachment Follows- >> {edited out by Ryan for brevity} >>
Re: correct? impossible to filter / facet on ExternalFileField
On Thu, Jun 11, 2009 at 12:53 PM, Britske wrote: > in our design some often changing fields would benefit from being defined as > ExternalFileFields, so we can index them more often than the rest. > > However we need to filter and facet on them. > I don't think that this currently is possible with ExternalFileField but > just want to make sure. > > if not possible, is it on the roadmap? You're right, it's not currently possible. We don't have any official roadmap, but we've thought about updateable fields (the other way to accomplish this sort of thing) and faceting/searching on an external source like ExternalFileField. -Yonik http://www.lucidimagination.com
correct? impossible to filter / facet on ExternalFileField
in our design some often changing fields would benefit from being defined as ExternalFileFields, so we can index them more often than the rest. However we need to filter and facet on them. I don't think that this currently is possible with ExternalFileField but just want to make sure. if not possible, is it on the roadmap? Thanks, Britske -- View this message in context: http://www.nabble.com/correct--impossible-to-filter---facet-on-ExternalFileField-tp23985106p23985106.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Upgrading 1.2.0 to 1.3.0 solr
I disagree with waiting that month. At this point, most of the kinks in the upgrade from 1.2 to 1.3 have been worked out. Waiting for 1.4 to come out risks you becoming a guinea pig for the upgrade procedure. Plus, if any show-stoppers come along delaying 1.4, you delay implementation of your auto-complete function. When 1.4 comes out, if it has any features you feel compel an upgrade, you can begin another round of testing and migration, but don't upgrade a production system just for the sake of being bleeding edge. Ryan T. Grange, IT Manager DollarDays International, Inc. rgra...@dollardays.com (480)922-8155 x106 Otis Gospodnetic wrote: Francis, If you can wait another month or so, you could skip 1.3.0, and jump to 1.4 which will be released soon. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch From: Francis Yakin To: "solr-user@lucene.apache.org" Sent: Wednesday, June 10, 2009 1:17:25 AM Subject: Upgrading 1.2.0 to 1.3.0 solr I am in process to upgrade our solr 1.2.0 to solr 1.3.0 Our solr 1.2.0 now is working fine, we just want to upgrade it cause we have an application that requires some function from 1.3.0( we call it autocomplete). Currently our config files on 1.2.0 are as follow: Solrconfig.xml Schema.xml ( we wrote this in house) Index_synonyms.txt ( we also modified and wrote this in house) Scripts.conf Protwords.txt Stopwords.txt Synonyms.txt I understand on 1.3.0 , it has new solrconfig.xml . My questions are: 1) what config files that I can reuse from 1.2.0 for 1.3.0 can I use the same schema.xml 2) Solrconfig.xml, can I use the 1.2.0 version or I have to stick with 1.3.0 If I need to stick with 1.3.0, what that I need to change. As of right I am testing it in my sandbox, so it doesn't work. Please advice, if you have any docs for upgrading 1.2.0 to 1.3.0 let me know. Thanks in advance Francis Note: I attached my solrconfigand schema.xml in this email -Inline Attachment Follows- {edited out by Ryan for brevity}
Getting details from
Hello, Is there any way to get the number of deleted records from a delete request? I'm sending: type_i:(2 OR 3) AND creation_time_rl:[0 TO 124426080] And getting: 02 This is Solr 1.3. -- Jacob Elder
Re: Faceting on text fields
Yao Ge schrieb: BTW, Carrot2 has a very impressive Clustering Workbench (based on eclipse) that has built-in integration with Solr. If you have a Solr service running, it is a just a matter of point the workbench to it. The clustering results and visualization are amazing. (http://project.carrot2.org/download.html). A new world opens up for me ... Thanks for pointing out how cool this is! Hint for other newcomers: Open the View Menu to configure the details of how you perform your search, e.g. your Solr URL in case it differs from the default, or your "summary field", which is what gets used to analyze the data in order to determine clusters, if I understand correctly. Michael Ludwig
Re: DataImportHandler backwards compatibility
On Thu, Jun 11, 2009 at 6:42 PM, Kevin Lloyd wrote: > > I'm in the process of implementing a DataImportHandler config for Solr 1.3 > and I've hit across the Postgresql/JDBC Out Of Memory problem. Whilst the > solution is documented on the wiki FAQ page: > > http://wiki.apache.org/solr/DataImportHandlerFaq > > it appears that the JDBC driver parameters were implemented in > DataImportHandler post the 1.3 release. Yes, those parameters are new in 1.4 (we should note that on the wiki). > I was wondering if it would be safe to take a nightly build of just the > DataImportHandler contrib and run it against a Solr 1.3 installation? > Solr 1.4 has a rollback command which 1.3 did not have. So, you'd need to hack the DataImportHandler code to remove references to RollBackCommand. You can use the 1.4 dih jar with 1.3 if you comment out the code in SolrWriter.rollback method, remove the import of RollbackUpdateCommand and recompile. -- Regards, Shalin Shekhar Mangar.
DataImportHandler backwards compatibility
Hi, I'm in the process of implementing a DataImportHandler config for Solr 1.3 and I've hit across the Postgresql/JDBC Out Of Memory problem. Whilst the solution is documented on the wiki FAQ page: http://wiki.apache.org/solr/DataImportHandlerFaq it appears that the JDBC driver parameters were implemented in DataImportHandler post the 1.3 release. I was wondering if it would be safe to take a nightly build of just the DataImportHandler contrib and run it against a Solr 1.3 installation? Many thanks, kevin
Re: Faceting on text fields
BTW, Carrot2 has a very impressive Clustering Workbench (based on eclipse) that has built-in integration with Solr. If you have a Solr service running, it is a just a matter of point the workbench to it. The clustering results and visualization are amazing. (http://project.carrot2.org/download.html). Yao Ge wrote: > > FYI. I did a direct integration with Carrot2 with Solrj with a separate > Ajax call from UI for top 100 hits to clusters terms in the two text > fields. It gots comparable performance to other facets in terms of > response time. > > In terms of algorithms, their listed two "Lingo" and "STC" which I don't > reconize. But I think at least one of them might have used SVD > (http://en.wikipedia.org/wiki/Singular_value_decomposition). > > -Yao > > > Otis Gospodnetic wrote: >> >> >> I'd call it related (their application in search encourages exploration), >> but also distinct enough to never mix them up. I think your assessment >> below is correct, although I'm not familiar with the details of Carrot2 >> any more (was once), so I can't tell you exactly which algo is used under >> the hood. >> >> Otis >> -- >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >> >> >> >> - Original Message >>> From: Michael Ludwig >>> To: solr-user@lucene.apache.org >>> Sent: Wednesday, June 10, 2009 9:41:54 AM >>> Subject: Re: Faceting on text fields >>> >>> Otis Gospodnetic schrieb: >>> > >>> > Solr can already cluster top N hits using Carrot2: >>> > http://wiki.apache.org/solr/ClusteringComponent >>> >>> Would it be fair to say that clustering as detailed on the page you're >>> referring to is a kind of dynamic faceting? The faceting not being done >>> based on distinct values of certain fields, but on the presence (and >>> frequency) of terms in one field? >>> >>> The main difference seems to be that with faceting, grouping criteria >>> (facets) are known beforehand, while with clustering, grouping criteria >>> (the significant terms which create clusters - the cluster keys) have >>> yet to be determined. Is that a correct assessment? >>> >>> Michael Ludwig >> >> >> > > -- View this message in context: http://www.nabble.com/Faceting-on-text-fields-tp23872891p23980959.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr update performance decrease after a while
2009/6/9 Vincent Pérès : > We are indexing approximately 500 documents per day. My benchmark says an > update is done in 0.7 sec just after Solr has been started. But it quickly > decrease to 2.2 secs per update ! Per document added, or for the whole update + commit? > I have just been focused on the Schema until now, and didn't changed many > stuffs in the solrconfig file. Maybe you have some tips which could help me > to be more linear ? Auto-warming (filling of caches based on previous queries) is enabled by default and happens with each commit. You may want to disable this (set autowarmCount=0 on the caches in solrconfig.xml). -Yonik http://www.lucidimagination.com
Re: Solr update performance decrease after a while
Thank for your answer, I will have a look and try to find a solution. Cheers -- View this message in context: http://www.nabble.com/Solr-update-performance-decrease-after-a-while-tp23945947p23980681.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Build Failed
Mukerjee, Neiloy (Neil) schrieb: Running ant -verbose still doesn't allow me to run an example configuration. I get the same error from "ant example" after getting the following from "ant -verbose": Build sequence for target(s) `usage' is [usage] usage: [echo] Welcome to the Solr project! [echo] Use 'ant example' to create a runnable example configuration. [echo] And for developers: [echo] Use 'ant clean' to clean compiled files. [echo] Use 'ant compile' to compile the source code. [echo] Use 'ant dist' to build the project WAR and JAR files. [echo] Use 'ant generate-maven-artifacts' to generate maven artifacts. [echo] Use 'ant package' to generate zip, tgz, and maven artifacts for distribution. [echo] Use 'ant test' to run unit tests. BUILD SUCCESSFUL You might want to read up on Ant usage in the Ant User Manual, a copy of which should be part of your installation, or can be found on the web. Quick overview: ant -help When I wrote "ant -verbose", I meant "ant -verbose ", so: ant -verbose example Michael Ludwig
Re: solr in distributed mode
Hi, i went through the document: http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr i have a couple of questions: 1. In the document its been mentioned that "There will be a 'master' server for each shard and then 1-n 'slaves' that are replicated from the master." how is the replication process done? suppose i have 2 machines nodeA and nodeB I edited scripts.config in solr/conf of both nodeA and nodeB to point to the master (i.e. nodeA). i) is it the right approach for setting up master/slave configuration? ii) to start the master/slave config, should i execute start.jar from both the nodes? or just from the master node? iii) are indexes automatically replicated when you insert/update it in the master.. or do we have to run a script for that? iv) how do i know if replication process is sucessfully carried out. v) suppose the master goes down. i do i perform a node failover.. for example make one of the slaves as master without disrupting my application? 2. It has also been mentioned that: "With distribution and replication, none of the master shards know about each other. You index to each master, the index is replicated to each slave, and then searches are distributed across the slaves, using one slave from each master/slave shard." i) Are slaves used only for index replications? i mean can't i have indexes distributed across slaves so that when i perform a search, it searches across all slaves? ii) since none of the shards have any information about one another, if i update/delete the document based on term, how does the index gets updated across all shards? or do we have to merge, update/delete and then distribute it across shards? Regards, Rakahi In a distributed configuration, one server 'shard' will get a query request and then search itself, as well as the other shards in the configuration, and return the combined results from each shard. On Wed, Jun 10, 2009 at 11:23 AM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > > Hello, > > All of this is covered on the Wiki, search for: distributed search > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message > > From: Rakhi Khatwani > > To: solr-user@lucene.apache.org > > Cc: ninad.r...@germinait.com; ranjit.n...@germinait.com; > saurabh.maha...@germinait.com > > Sent: Tuesday, June 9, 2009 4:55:55 AM > > Subject: solr in distributed mode > > > > Hi, > > I was looking for ways in which we can use solr in distributed mode. > > is there anyways we can use solr indexes across machines or by using > Hadoop > > Distributed File System? > > > > Its has been mentioned in the wiki that > > When an index becomes too large to fit on a single system, or when a > single > > query takes too long to execute, an index can be split into multiple > shards, > > and Solr can query and merge results across those shards. > > > > what i understand is that shards are a partition. are shards on the same > > machine or can it be on different machines?? do we have to manually > > split the indexes to store in different shards. > > > > do you have an example or some tutorial which demonstrates distributed > index > > searching/ storing using shards? > > > > Regards, > > Raakhi > >
Re: Build Failed
On Thu, Jun 11, 2009 at 5:49 PM, Mukerjee, Neiloy (Neil) < neil.muker...@alcatel-lucent.com> wrote: > Running ant -verbose still doesn't allow me to run an example > configuration. I get the same error from "ant example" after getting the > following from "ant -verbose": > > Apache Ant version 1.7.0 compiled on August 29 2007 > Buildfile: build.xml > Detected Java version: 1.6 in: /usr/lib/jvm/java-1.5.0-gcj-4.2-1.5.0.0/jre > You are using GCJ. Switch to Sun JDK. -- Regards, Shalin Shekhar Mangar.
Re: Search Phrase Wildcard?
You might be interested in this Lucene issue: https://issues.apache.org/jira/browse/LUCENE-1486 Aleksander M. Stensby wrote: Well yes:) Since Solr do infact support the entire lucene query parser syntax:) - Aleks On Thu, 11 Jun 2009 13:57:23 +0200, Avlesh Singh wrote: Infact, Lucene does not support that. Lucene supports single and multiple character wildcard searches within single terms (*not within phrase queries*). Taken from http://lucene.apache.org/java/2_3_2/queryparsersyntax.html#Wildcard%20Searches Cheers Avlesh On Thu, Jun 11, 2009 at 4:32 PM, Aleksander M. Stensby < aleksander.sten...@integrasco.no> wrote: Solr does not support wildcards in phrase queries, yet. Cheers, Aleks On Thu, 11 Jun 2009 11:48:13 +0200, Samnang Chhun wrote: Hi all, I have my document like this: Solr web service Is there any ways that I can search like startswith: "So* We*" : found "Sol*": found "We*": not found Cheers, Samnang -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail -- - Mark http://www.lucidimagination.com
RE: Build Failed
Running ant -verbose still doesn't allow me to run an example configuration. I get the same error from "ant example" after getting the following from "ant -verbose": Apache Ant version 1.7.0 compiled on August 29 2007 Buildfile: build.xml Detected Java version: 1.6 in: /usr/lib/jvm/java-1.5.0-gcj-4.2-1.5.0.0/jre Detected OS: Linux parsing buildfile /home/stagger2/Solr/apache-solr-1.3.0/build.xml with URI = file:/home/stagger2/Solr/apache-solr-1.3.0/build.xml Project base dir set to: /home/stagger2/Solr/apache-solr-1.3.0 [antlib:org.apache.tools.ant] Could not load definitions from resource org/apache/tools/ant/antlib.xml. It could not be found. Importing file /home/stagger2/Solr/apache-solr-1.3.0/common-build.xml from /home/stagger2/Solr/apache-solr-1.3.0/build.xml parsing buildfile /home/stagger2/Solr/apache-solr-1.3.0/common-build.xml with URI = file:/home/stagger2/Solr/apache-solr-1.3.0/common-build.xml [property] Loading /home/stagger2/Solr/apache-solr-1.3.0/build.properties [property] Unable to find property file: /home/stagger2/Solr/apache-solr-1.3.0/build.properties Override ignored for property "DSTAMP" Override ignored for property "TSTAMP" [available] Unable to load class org.apache.maven.artifact.ant.Pom to set property maven.ant.tasks.present [available] Unable to load class com.cenqua.clover.tasks.CloverReportTask to set property clover.present [macrodef] creating macro solr-javac [macrodef] creating macro solr-jar [macrodef] creating macro solr-checksum [macrodef] creating macro contrib-crawl [macrodef] creating macro invoke-javadoc Property "pom.xml" has not been set Property "jar.file" has not been set [macrodef] creating macro m2-deploy [available] Unable to load class com.cenqua.clover.tasks.CloverReportTask to set property clover.present Build sequence for target(s) `usage' is [usage] Complete build sequence is [usage, common.make-manifest, init-forrest-entities, compile-common, compile, compile-solrj-core, compile-solrj, compileTests, junit, test-core, build-contrib, test-contrib, test, make-manifest, dist-jar, dist-solrj, dist-contrib, dist-war, dist, example, javadoc-core, javadoc-common, javadoc-contrib, javadoc-solrj, javadoc-all, javadoc, dist-src, dist-javadoc, create-package, nightly, clover.setup, common.dist-contrib, clean-contrib, clean, svn-up, build-site, maven.ant.tasks-check, generate-maven-artifacts, package, prepare-release, clean-dest, common.test-contrib, -taskdef, stub-factories, rat-sources, test-reports, clover.info, clover.check, clover, generate-clover-reports, common.maven.ant.tasks-check, common.build-contrib, common.clean-contrib, ] usage: [echo] Welcome to the Solr project! [echo] Use 'ant example' to create a runnable example configuration. [echo] And for developers: [echo] Use 'ant clean' to clean compiled files. [echo] Use 'ant compile' to compile the source code. [echo] Use 'ant dist' to build the project WAR and JAR files. [echo] Use 'ant generate-maven-artifacts' to generate maven artifacts. [echo] Use 'ant package' to generate zip, tgz, and maven artifacts for distribution. [echo] Use 'ant test' to run unit tests. BUILD SUCCESSFUL Total time: 1 second -Original Message- From: Michael Ludwig [mailto:m...@as-guides.com] Sent: Thursday, June 11, 2009 6:05 AM To: solr-user@lucene.apache.org Subject: Re: Build Failed Mukerjee, Neiloy (Neil) schrieb: > When running "ant example" to do an example configuration, I get the > following message: > > BUILD FAILED > /home/stagger2/Solr/apache-solr-1.3.0/common-build.xml:149: Compile > failed; see the compiler error output for details. > > I've tried reading through the files in question, but I can't seem to > find the issue. Any suggestions? Run: ant -verbose Michael Ludwig
Re: Faceting on text fields
FYI. I did a direct integration with Carrot2 with Solrj with a separate Ajax call from UI for top 100 hits to clusters terms in the two text fields. It gots comparable performance to other facets in terms of response time. In terms of algorithms, their listed two "Lingo" and "STC" which I don't reconize. But I think at least one of them might have used SVD (http://en.wikipedia.org/wiki/Singular_value_decomposition). -Yao Otis Gospodnetic wrote: > > > I'd call it related (their application in search encourages exploration), > but also distinct enough to never mix them up. I think your assessment > below is correct, although I'm not familiar with the details of Carrot2 > any more (was once), so I can't tell you exactly which algo is used under > the hood. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >> From: Michael Ludwig >> To: solr-user@lucene.apache.org >> Sent: Wednesday, June 10, 2009 9:41:54 AM >> Subject: Re: Faceting on text fields >> >> Otis Gospodnetic schrieb: >> > >> > Solr can already cluster top N hits using Carrot2: >> > http://wiki.apache.org/solr/ClusteringComponent >> >> Would it be fair to say that clustering as detailed on the page you're >> referring to is a kind of dynamic faceting? The faceting not being done >> based on distinct values of certain fields, but on the presence (and >> frequency) of terms in one field? >> >> The main difference seems to be that with faceting, grouping criteria >> (facets) are known beforehand, while with clustering, grouping criteria >> (the significant terms which create clusters - the cluster keys) have >> yet to be determined. Is that a correct assessment? >> >> Michael Ludwig > > > -- View this message in context: http://www.nabble.com/Faceting-on-text-fields-tp23872891p23980124.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Search Phrase Wildcard?
Well yes:) Since Solr do infact support the entire lucene query parser syntax:) - Aleks On Thu, 11 Jun 2009 13:57:23 +0200, Avlesh Singh wrote: Infact, Lucene does not support that. Lucene supports single and multiple character wildcard searches within single terms (*not within phrase queries*). Taken from http://lucene.apache.org/java/2_3_2/queryparsersyntax.html#Wildcard%20Searches Cheers Avlesh On Thu, Jun 11, 2009 at 4:32 PM, Aleksander M. Stensby < aleksander.sten...@integrasco.no> wrote: Solr does not support wildcards in phrase queries, yet. Cheers, Aleks On Thu, 11 Jun 2009 11:48:13 +0200, Samnang Chhun wrote: Hi all, I have my document like this: Solr web service Is there any ways that I can search like startswith: "So* We*" : found "Sol*": found "We*": not found Cheers, Samnang -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Re: Search Phrase Wildcard?
Infact, Lucene does not support that. Lucene supports single and multiple character wildcard searches within > single terms (*not within phrase queries*). > Taken from http://lucene.apache.org/java/2_3_2/queryparsersyntax.html#Wildcard%20Searches Cheers Avlesh On Thu, Jun 11, 2009 at 4:32 PM, Aleksander M. Stensby < aleksander.sten...@integrasco.no> wrote: > Solr does not support wildcards in phrase queries, yet. > > Cheers, > Aleks > > > On Thu, 11 Jun 2009 11:48:13 +0200, Samnang Chhun > wrote: > > Hi all, >> I have my document like this: >> >> >> >> Solr web service >> >> >> >> Is there any ways that I can search like startswith: >> >> "So* We*" : found >> "Sol*": found >> "We*": not found >> >> Cheers, >> Samnang >> > > > > -- > Aleksander M. Stensby > Lead software developer and system architect > Integrasco A/S > www.integrasco.no > http://twitter.com/Integrasco > > Please consider the environment before printing all or any of this e-mail >
Re: Search Phrase Wildcard?
Solr does not support wildcards in phrase queries, yet. Cheers, Aleks On Thu, 11 Jun 2009 11:48:13 +0200, Samnang Chhun wrote: Hi all, I have my document like this: Solr web service Is there any ways that I can search like startswith: "So* We*" : found "Sol*": found "We*": not found Cheers, Samnang -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Re: dismax parsing applied to specific fields
Nick Jenkin schrieb: Hi I was wondering if there is a way of applying dismax parsing to specific fields, where there are multiple fields being searched - all with different query values e.g. author:(tolkien) AND title:(the lord of the rings) would be something like: dismax(author, tolkien) AND dismax(title, the lord of the rings) I guess this can be thought of having two separate dismax configurations, one searching author and one searching title - and the intersection of the results is returned. http://wiki.apache.org/solr/DisMaxRequestHandler This says that "the DisMaxRequestHandler is simply the standard request handler with the default query parser set to the DisMax Query Parser". So maybe you could program your own CustomDisMaxRequestHandler that reuses the DisMax query parser (and probably other components) to achieve what you want. Michael Ludwig
Re: How to use offset and limit in Solr
Thanks On Thu, Jun 11, 2009 at 3:29 PM, Markus Jelsma - Buyways B.V. < mar...@buyways.nl> wrote: > Use start and rows instead. See the common query parameters > > http://wiki.apache.org/solr/CommonQueryParameters > > - > Markus Jelsma Buyways B.V. Tel. 050-3118123 > Technisch ArchitectFriesestraatweg 215c Fax. 050-3118124 > http://www.buyways.nl 9743 AD GroningenKvK 01074105 > > > On Thu, 2009-06-11 at 15:23 +0700, chem leakhina wrote: > > > Hi > > > > How can I use offset or limit in Solr? > > Could you give some examples? > > > > Best regards, > > LEE >
Re: Build Failed
Mukerjee, Neiloy (Neil) schrieb: When running "ant example" to do an example configuration, I get the following message: BUILD FAILED /home/stagger2/Solr/apache-solr-1.3.0/common-build.xml:149: Compile failed; see the compiler error output for details. I've tried reading through the files in question, but I can't seem to find the issue. Any suggestions? Run: ant -verbose Michael Ludwig
Re: Customizing results
revas schrieb: What is GNU gettext and how this can be used in a multilanguage scenario? It'a an internationalization technology, so it is well suited to the tasks of internationalizing and localizing applications. http://www.gnu.org/software/gettext/manual/ http://www.gnu.org/software/gettext/manual/html_node/Why.html In your case, it might mean that the client is equipped with the language packages it needs and uses the name returned by Solr (likely the English term) to look up the translation by means of Gettext. But it certainly depends very much on your particular setup. It might be overkill for your particular situation. Michael Ludwig
Search Phrase Wildcard?
Hi all, I have my document like this: Solr web service Is there any ways that I can search like startswith: "So* We*" : found "Sol*": found "We*": not found Cheers, Samnang
Re: How to use offset and limit in Solr
Query Parameters: start=yourOffsetValue&rows=yourLimitValue If you use a SolrJ client: SolrQuery query = new SolrQuery(); query.setQuery(yourQuery); query.setRows(yourLimitValue); query.setStart(yourOffsetValue); Cheers Avlesh On Thu, Jun 11, 2009 at 1:53 PM, chem leakhina wrote: > Hi > > How can I use offset or limit in Solr? > Could you give some examples? > > Best regards, > LEE >
Re: How to use offset and limit in Solr
Use start and rows instead. See the common query parameters http://wiki.apache.org/solr/CommonQueryParameters - Markus Jelsma Buyways B.V. Tel. 050-3118123 Technisch ArchitectFriesestraatweg 215c Fax. 050-3118124 http://www.buyways.nl 9743 AD GroningenKvK 01074105 On Thu, 2009-06-11 at 15:23 +0700, chem leakhina wrote: > Hi > > How can I use offset or limit in Solr? > Could you give some examples? > > Best regards, > LEE
How to use offset and limit in Solr
Hi How can I use offset or limit in Solr? Could you give some examples? Best regards, LEE
Re: How to copyFeild to reverse string in another field
Hmm, you want to do that because sorl doesn't allow* and ? as the first character? On Jun 11, 2009, at 9:02 AM, chem leakhina wrote: Hi How can I use copyField to copy field that contains normal text to another field and that field contains as reversed text? Best regards, Lee -- Reza Safari LUKKIEN Copernicuslaan 15 6716 BM Ede The Netherlands - http://www.lukkien.com t: +31 (0) 318 698000 This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.
Re: How to support stemming and case folding for english content mixed with non-english content?
Note: I request Solr users to go through this mail and let me thier ideas. Thanks Yonik, you rightly pointed it out. That clearly says that the way I'm trying to mimic the default behaviour of Solr indexing/searching in Lucene is wrong, right?. I downloaded the latest version of solr nightly on may20[at that time I was using Solr, now switched to Lucene]. I hope the issue must have been fixed with that version.Anyway I'm going to download the latest nightly build today and try it out. I hope using the nightly build instead of getting the src from latest trunk is more or less same[provided I donwload the latest nightly build, right?]as I don't know much about getting/compiling the src from solr trunk. Do let me know if I've to use the trunk anyway, in that case I'm ready to spend time to get that done. BTW, Yonik, as per the basic Solr schema.xml file, the analyzers/filters used by default are these ones, correct me if I'm wrong, this is the code snip that mentions the filters used for indexing in Solr and this is the part used for Solr querying, To summarize the names are like this, Indexing: 1. solr.WhitespaceTokenizerFactory -- tokenizer and the followings filters as is clear from the name itself 2. solr.SynonymFilterFactory 3. solr.StopFilterFactory 4. solr.WordDelimiterFilterFactory (with the options as, generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1") 5. solr.LowerCaseFilterFactory 6. solr.EnglishPorterFilterFactory 7. solr.RemoveDuplicatesTokenFilterFactory Querying: 1. solr.WhitespaceTokenizerFactory 2. solr.SynonymFilterFactory 3. solr.StopFilterFactory 4. solr.WordDelimiterFilterFactory( options are: generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1") 5. solr.LowerCaseFilterFactory 6. solr.EnglishPorterFilterFactory 7. solr.RemoveDuplicatesTokenFilterFactory Now the filters/analyzers I used that tried to mimic the above behavior of Solr [in Lucene] is as show below. I pulled out the whitespacedelimiterfilter from Solr and my custom analyzer for indexing is like this, /** * Analyzer for Indian language. */ public class IndicAnalyzerIndex extends Analyzer { public TokenStream tokenStream(String fieldName, Reader reader) { TokenStream ts = new WhitespaceTokenizer(reader); ts = new WordDelimiterFilter(ts, 1, 1, 1, 1, 0); // I tried using ...(ts, 1, 1, 1, 1, 0, 1) 7 params, but no constructor found for that, I didn't try to modify the code to add this feature though, then used this with 6 params, that uses the constructor for which the last option for splitOnCaseChange is set to 1 so we're doing the same thing even in this way... ts = new StopFilter(ts, StopAnalyzer.ENGLISH_STOP_WORDS); ts = new LowerCaseFilter(ts); ts = new PorterStemFilter(ts); return ts; } } and for querying this is teh code /** * Analyzer for Indian language. */ public class IndicAnalyzerQuery extends Analyzer { public TokenStream tokenStream(String fieldName, Reader reader) { TokenStream ts = new WhitespaceTokenizer(reader); ts = new WordDelimiterFilter(ts, 1, 1, 0, 0, 0); ts = new StopFilter(ts, StopAnalyzer.ENGLISH_STOP_WORDS); ts = new LowerCaseFilter(ts); ts = new PorterStemFilter(ts); return ts; } } The only difference for both is just the worddelimiterfilter with different options... Comparing the analyzers/filters used by Solr and the above custom analyzer we can see that I'm not using synonymfilter and removeduplicatefilter. I hope these make sense for english content only and using/skipping them will not make any differece to my non-english content. Can someone with knowledge of Solr/Lulcene source code point me what exactly is going wrong in my case whn I'm trying to do the same thing in Lucene. It seems I'm missing some minor yet important thing...hence my custom IndicAnalyzer is not behaving the way Solr's default anlyzer works and this is clearly shown by Yonik that Solr is smart enough to detect unicoded word endings and behaving as expected. Any idea on this issue is welcome. Help me fix the issue. BTW, lucene ppl when is that basic worddelimiterfilter going to be added to Lucene as well? Any idea? Thanks, KK. On Tue, Jun 9, 2009 at 7:01 PM, Yonik Seeley wrote: > I just cut'n'pasted your word into Solr... it worked fine (it didn't > split the word). > Make sure you're using the latest from the trunk version of Solr... > this was fixed since 1.3 > > http://localhost:8983/solr/select?q=साल&debugQuery=true > [...] > > साल > साल > text:साल > text:साल > > -Yonik > > > On Tue, Jun 9, 2009 at 7:48 AM, KK wrote: > > Hi Robert, I tried a sample code to check wha
How to copyFeild to reverse string in another field
Hi How can I use copyField to copy field that contains normal text to another field and that field contains as reversed text? Best regards, Lee