Re: Sum of one field
The easiest solution would probably be to have a facet on the quantity field and calculate the total quantity on the client side. Svein On 4. aug.. 2008, at 21.47, Otis Gospodnetic wrote: Leonardo, You'd have to read that quantity fields for all matching documents one way or the other. One way is by getting all results and pulling that field out, so you can get the sum.. Another way is to hack the SolrIndexSearcher and get this value in one of the HitCollector collect method calls. Another possibility, if your index is fairly static, might be to read it all documents' (not just matches') quantity field and store that in a docID-quantity map structure that lets you look up quantity for any docID you want. There may be other/better ways of doing this, but this is what comes to (my) mind first. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Leonardo Dias [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Monday, August 4, 2008 1:19:45 PM Subject: Sum of one field Everyone exhibits your search for x has returned y results on the top of the results page, but we need something else, which would be something like your search for x returned y results in z records, being z the numdocs of the SOLR response and y a SUM(quantity) of all returned records. In SQL you can do something like: SELECT count(1), sum(quantity) FROM table But with SOLR we don't know how can we do the same without having to return all the XML result for the field quantity and then sum it to show the total. Any hints on how to do it in a better way? cheers, Leonardo
localsolr and dataimport problems
I'm trying to use localsolr (1.5) with the dataimport handler but not having much luck. The issue seems to be that the _localtier dynamic fields dont get generated when adding docs via the dataimport, although they do if I add docs via post.jar (xml document). Am i missing something simple here? -- View this message in context: http://www.nabble.com/localsolr-and-dataimport-problems-tp18849983p18849983.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Logo thought
On Tue, 05 Aug 2008 16:02:51 -0400
Stephen Weiss [EMAIL PROTECTED] wrote:
My issue with the logos presented was they made solr look like a
school project instead of the powerful tool that it is. The tricked
out font or whatever just usually doesn't play well with the business
types... they want serious-looking software. First impressions are
everything. While the fiery colors are appropriate for something
named Solr, you can play with that without getting silly - take a look
at:
couldn't agree more. current logo needs improvement, but I think it can be done
much better... In particular thinking of small icons, print,etc...
http://www.ascsolar.com/images/asc_solar_splash_logo.gif
http://www.logostick.com/images/EOS_InvestmentingLogo_lg.gif
(Luckily there are many businesses that do solar energy!)
They have the same elements but with a certain simplicity and elegance.
I know probably some people don't care if it makes the boss or client
happy, but, these are the kinds of seemingly insignificant things that
Indeed - the way I see it, if you don't care either way, then you should be
happy to have a professional looking one :P
B
_
{Beto|Norberto|Numard} Meijome
Caminante no hay camino, se hace camino al andar
Antonio Machado
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
Re: Sum of one field
Hi, Svein! Your approach is a great idea, even though it is not the best yet. It would be even nicer if we could just tell SOLR to sum up everything in a facet, already caching that... but this is great and it could work nicely! Thank you very much. We will try that out. This is more of a solr-dev question, but I don't subscribe to that list, so here it goes: where in the SOLR code we could change that facet behaviour to cope with this idea or a much better one? We have resources here. With the right hints we could even contribute with SOLR improvement. Best, Leonardo Svein Parnas escreveu: The easiest solution would probably be to have a facet on the quantity field and calculate the total quantity on the client side. Svein On 4. aug.. 2008, at 21.47, Otis Gospodnetic wrote: Leonardo, You'd have to read that quantity fields for all matching documents one way or the other. One way is by getting all results and pulling that field out, so you can get the sum.. Another way is to hack the SolrIndexSearcher and get this value in one of the HitCollector collect method calls. Another possibility, if your index is fairly static, might be to read it all documents' (not just matches') quantity field and store that in a docID-quantity map structure that lets you look up quantity for any docID you want. There may be other/better ways of doing this, but this is what comes to (my) mind first. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Leonardo Dias [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Monday, August 4, 2008 1:19:45 PM Subject: Sum of one field Everyone exhibits your search for x has returned y results on the top of the results page, but we need something else, which would be something like your search for x returned y results in z records, being z the numdocs of the SOLR response and y a SUM(quantity) of all returned records. In SQL you can do something like: SELECT count(1), sum(quantity) FROM table But with SOLR we don't know how can we do the same without having to return all the XML result for the field quantity and then sum it to show the total. Any hints on how to do it in a better way? cheers, Leonardo
Re: Sum of one field
On Wed, Aug 6, 2008 at 10:46 AM, Koji Sekiguchi [EMAIL PROTECTED] wrote: Just on I'm working on the similar issue per customer request. StatsComponent - it will return min,max,sum,qt,avg as follows: Seems like perhaps one should be able to return any arbitrary function (actually multiple), and sort by an arbitrary function also. -Yonik stats=onstats.field=price stats stats_fields lst name=price double name=min10/double double name=max30/double double name=avg20/double double name=sum60/double double name=qt3/double /lst /stats_fields /stats WRT stats, the component can output sum and avg, but not sd and var. As our main purpose is to get market price, the output should be enough. I'll upload the patch soon. Koji
RE: config reload JMX capabilities
Thanks Ryan. For the time being this would probably work for us. However is there any plan in the future to expose a JMX MBean to perform this operation? -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 06, 2008 5:45 AM To: solr-user@lucene.apache.org Subject: Re: config reload JMX capabilities I don't know about JMX, but check the standard multi-core config... If you are running things in multi-core mode, you can send a RELOAD command: http://wiki.apache.org/solr/MultiCore#head-429a06cb83e1ce7b06857fd03c38d 1200c4bcfc1 On Aug 5, 2008, at 2:39 PM, Kashyap, Raghu wrote: One of the requirements we have is that when we deploy new data for solr config (synonyms, dictionary etc) we should NOT be restarting the solr instances for the changes to take effect. Are there ConfigReload capabilities through JMX that can help us do this? Thanks in Advance -Raghu
RE: config reload JMX capabilities
Thanks Shalin. I am watching the JIRA since that is something we are very interested in (Replication through JMX). -Raghu -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 05, 2008 11:59 PM To: solr-user@lucene.apache.org Subject: Re: config reload JMX capabilities On Wed, Aug 6, 2008 at 3:09 AM, Kashyap, Raghu [EMAIL PROTECTED]wrote: Are there ConfigReload capabilities through JMX that can help us do this? No, only statistics are exposed through JMX at present. SOLR-561 enables support for automatic config file replication to slaves without downtime. However, a lot of work is left in that. https://issues.apache.org/jira/browse/SOLR-561 -- Regards, Shalin Shekhar Mangar.
Re: Sum of one field
Just on I'm working on the similar issue per customer request. StatsComponent - it will return min,max,sum,qt,avg as follows: Seems like perhaps one should be able to return any arbitrary function (actually multiple), and sort by an arbitrary function also. Sounds interesting. WRT an arbitrary function, do you mean: stats stats_fields lst name=price double name=func110/double double name=func230/double double name=func320/double /lst /stats_fields /stats But support of an arbitrary function should be so complex on distributed environment? In the meanwhile I'll implement stats component to fullfill our needs. :) Koji
Re: localsolr and dataimport problems
On Wed, Aug 6, 2008 at 5:50 PM, TomWilliamson [EMAIL PROTECTED]wrote: I'm trying to use localsolr (1.5) with the dataimport handler but not having much luck. The issue seems to be that the _localtier dynamic fields dont get generated when adding docs via the dataimport, although they do if I add docs via post.jar (xml document). Am i missing something simple here? -- View this message in context: http://www.nabble.com/localsolr-and-dataimport-problems-tp18849983p18849983.html Sent from the Solr - User mailing list archive at Nabble.com. Hmm... LocalSolr uses the UpdateRequestProcessor API in Solr to add it's dynamic fields. However, DataImportHandler bypasses that API and adds documents directly to the UpdateHandler. Sounds like an improvement (bug?) in DataImportHandler which should be addressed before release. I'll open an issue and work on it. -- Regards, Shalin Shekhar Mangar.
Re: localsolr and dataimport problems
I've opened SOLR-647 to fix this. https://issues.apache.org/jira/browse/SOLR-676 On Wed, Aug 6, 2008 at 9:56 PM, Shalin Shekhar Mangar [EMAIL PROTECTED] wrote: On Wed, Aug 6, 2008 at 5:50 PM, TomWilliamson [EMAIL PROTECTED] wrote: I'm trying to use localsolr (1.5) with the dataimport handler but not having much luck. The issue seems to be that the _localtier dynamic fields dont get generated when adding docs via the dataimport, although they do if I add docs via post.jar (xml document). Am i missing something simple here? -- View this message in context: http://www.nabble.com/localsolr-and-dataimport-problems-tp18849983p18849983.html Sent from the Solr - User mailing list archive at Nabble.com. Hmm... LocalSolr uses the UpdateRequestProcessor API in Solr to add it's dynamic fields. However, DataImportHandler bypasses that API and adds documents directly to the UpdateHandler. Sounds like an improvement (bug?) in DataImportHandler which should be addressed before release. I'll open an issue and work on it. -- Regards, Shalin Shekhar Mangar. -- Regards, Shalin Shekhar Mangar.
Re: localsolr and dataimport problems
My mistake, it is SOLR-676. https://issues.apache.org/jira/browse/SOLR-676 On Thu, Aug 7, 2008 at 12:13 AM, Shalin Shekhar Mangar [EMAIL PROTECTED] wrote: I've opened SOLR-647 to fix this. https://issues.apache.org/jira/browse/SOLR-676 On Wed, Aug 6, 2008 at 9:56 PM, Shalin Shekhar Mangar [EMAIL PROTECTED] wrote: On Wed, Aug 6, 2008 at 5:50 PM, TomWilliamson [EMAIL PROTECTED] wrote: I'm trying to use localsolr (1.5) with the dataimport handler but not having much luck. The issue seems to be that the _localtier dynamic fields dont get generated when adding docs via the dataimport, although they do if I add docs via post.jar (xml document). Am i missing something simple here? -- View this message in context: http://www.nabble.com/localsolr-and-dataimport-problems-tp18849983p18849983.html Sent from the Solr - User mailing list archive at Nabble.com. Hmm... LocalSolr uses the UpdateRequestProcessor API in Solr to add it's dynamic fields. However, DataImportHandler bypasses that API and adds documents directly to the UpdateHandler. Sounds like an improvement (bug?) in DataImportHandler which should be addressed before release. I'll open an issue and work on it. -- Regards, Shalin Shekhar Mangar. -- Regards, Shalin Shekhar Mangar. -- Regards, Shalin Shekhar Mangar.
Re: Loading solr from .properties file
This should be one use-case for https://issues.apache.org/jira/browse/SOLR-646 SOLR-646 . If you can try it, don't hesitate to report/comment on the issue. Henri zayhen wrote: Hello guys, I have to load solr/home from a .properties file, because of some environment standards I have to follow for one client that insists I should deliver Solr in one .ear containing its .war The thing is: the same .ear in testing must the .ear in production, so I can't change the env-entry in web.xml The other problem, the sysadmin won't let me alter JVM parameters. I would like to rear the experts opinion. -- Alexander Ramos Jardim - RPG da Ilha -- View this message in context: http://www.nabble.com/Loading-solr-from-.properties-file-tp18851924p18857371.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Loading solr from .properties file
Hello Henrib, I have read the issue and it seems an interesting feature for Solr, but I don't see how it address to my needs, as I need to point Solr to the multicore.properties file. Actually, I have already resolved my problem, hacking SolrResourceLoader so that it can load a solr.properties file inside a directory defined in a standard JVM argument the sysadmin puts on all JVM's (I know this is stupid. He could put the solr.home, but NO, he said! Use the standards! F0cking sysadmin...) 2008/8/6 Henrib [EMAIL PROTECTED] This should be one use-case for https://issues.apache.org/jira/browse/SOLR-646 SOLR-646 . If you can try it, don't hesitate to report/comment on the issue. Henri zayhen wrote: Hello guys, I have to load solr/home from a .properties file, because of some environment standards I have to follow for one client that insists I should deliver Solr in one .ear containing its .war The thing is: the same .ear in testing must the .ear in production, so I can't change the env-entry in web.xml The other problem, the sysadmin won't let me alter JVM parameters. I would like to rear the experts opinion. -- Alexander Ramos Jardim - RPG da Ilha -- View this message in context: http://www.nabble.com/Loading-solr-from-.properties-file-tp18851924p18857371.html Sent from the Solr - User mailing list archive at Nabble.com. -- Alexander Ramos Jardim
Re: Restricting spellchecker for certain words
: It seems that spellchecker works great except all the 7 words you can't say : on TV resolve to very important people, is there a way to contain just : certain words so they don't resolve? I'm by no means an expert on the current machinations of the spellchecker component/handler code, but there was work being done along the lines of letting people customize how query strings were processed before the spelling lookup was done. skimming the wiki it looks like you could create a QueryConverter and configure it to ignore any words you want (or to use an analyzer with a stopword filter for example) http://wiki.apache.org/solr/SpellCheckComponent -Hoss
Re: Loading solr from .properties file
My bad, sorry, I read too fast and skipped the fist necessary which is to get solr.home through JNDI which you can't have nor set I presume... (which from there would allow reading a multicore.properties, etc...) I can only hope you are not running on Websphere (where the same kind of sysadmin go ballistic when they see logs in their console...) Cheers Henrib zayhen wrote: Hello Henrib, I have read the issue and it seems an interesting feature for Solr, but I don't see how it address to my needs, as I need to point Solr to the multicore.properties file. Actually, I have already resolved my problem, hacking SolrResourceLoader so that it can load a solr.properties file inside a directory defined in a standard JVM argument the sysadmin puts on all JVM's (I know this is stupid. He could put the solr.home, but NO, he said! Use the standards! F0cking sysadmin...) -- View this message in context: http://www.nabble.com/Loading-solr-from-.properties-file-tp18851924p18860185.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: faceting and unique ids
I'm afraid i really don't understand your questions, particularly this part... : david was faceted as 3... how one could map any of these personids to the right david ?? as i understand the info you've sent, you have 4 docs in your results and 3 of them have a firstname field containing david and 1 of them has a firstname field containing dylan. that's what the faceting info tells you. if you want to know *which* 3 contain david, you either need to store the firstname field and include in in your fl, or you need to explicitly search for firstname:david. Based on your statements, it sound like you know that the number 1 in your personid field corrisponds with 'david', but Solr has no way of knowing that. -Hoss
Re: faceting and unique ids
david was faceted as 3... how one could map any of these personids to the right david ?? lst name=facet_fieldslst name=firstnameint name=david3/int... 3 is _not_ presonId, it is count of persons whose name is David...
RE: Solr* != solr*
: have solved it in their client. I can't modify Solr / Lucene code and I'm : using Solr 1.2. : : What I have done is simple. Given a user input, I break it into words and : then analyze each word. Any word contains wildcards (* Or ?) I lowercase : it. : : While the logic is simple, I'm not comfortable with it because the : word-breaker isn't based on the analyzer in use by Lucene. In my case, I : can't tell which analyzer is used. i wouldn't relaly owrry baout that -- the QueryParser doesn't know which Analyzer to use until it's done a first pass over the input and split it up into QueryParser.Tokens (not to be confused with o.a.l.analysis.Tokens) as well. Depending on how worried you are about things like quote characters appearing in your user input, splitting on whitespace is essentially what the Lucene QueryParser does as well .. if you really want to be carefull about things, you can use a custom subclass of the QueryParser on your client side, and let the query parser grammer split the input up for you. (disclaimer: i've never tried doing anything remotely like this) -Hoss
Re: Unlock on startup
: unlockOnStartuptrue/unlockOnStartup. : But restarting my server still doesn't seem to remove the writelock : file. Is there some other configuration that I might have to do get this : fixed. I can reproduce these symptoms, are you seeing an exception like the one i posted here.. http://issues.apache.org/jira/browse/SOLR-677 ? -Hoss
case preserving for data but not for indexing
In order to preserve case for the data, but not for indexing, I have created two fields. One is type Author that is defined as: fieldType name=author class=solr.StrField sortMissingLast=true omitNorms=true analyzer tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.StandardTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType and the other is just string: fieldType name=string class=solr.StrField sortMissingLast=true omitNorms=true/ this is used then for the author lists: field name=authors type=author indexed=true stored=true omitNorms=true multiValued=true/ field name=all_authors type=string indexed=true stored=true omitNorms=true multiValued=true/ Is there any other way than to have two fields like this? One for searching and one for displaying. People's names can be vary case sensitive for display purpose (eg McDonald. DeBros) but I don't want people to miss results because they search for lee instead of Lee. Also, can anyone see danger is using StandardTokenizerFactory for people's names? -- Regards, Ian Connor
Re: Unlock on startup
What date did you get 1.3-dev? I committed a patch for this on Jul 12th. -Yonik On Tue, Aug 5, 2008 at 1:38 PM, sundar shankar [EMAIL PROTECTED] wrote: Hi All, I am having to test solr indexing quite a bit on my local and dev environments. I had the unlockOnStartuptrue/unlockOnStartup. But restarting my server still doesn't seem to remove the writelock file. Is there some other configuration that I might have to do get this fixed. My Configurations : Solr 1.3 on Windows xp(local) and RHL on dev box. Jboss 4.05 Regards Sundar _ Searching for the best deals on travel? Visit MSN Travel. http://msn.coxandkings.co.in/cnk/cnk.do
Re: case preserving for data but not for indexing
Maybe I'm missing something (it's late) but why not just index+store? The stored value will be the original and indexing can lowercase (as you set it), so it's case-insensitive. Also, does this actually work for you: tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.StandardTokenizerFactory/ 2 Tokenizers? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Ian Connor [EMAIL PROTECTED] To: solr solr-user@lucene.apache.org Sent: Wednesday, August 6, 2008 8:21:28 PM Subject: case preserving for data but not for indexing In order to preserve case for the data, but not for indexing, I have created two fields. One is type Author that is defined as: sortMissingLast=true omitNorms=true and the other is just string: sortMissingLast=true omitNorms=true/ this is used then for the author lists: omitNorms=true multiValued=true/ stored=true omitNorms=true multiValued=true/ Is there any other way than to have two fields like this? One for searching and one for displaying. People's names can be vary case sensitive for display purpose (eg McDonald. DeBros) but I don't want people to miss results because they search for lee instead of Lee. Also, can anyone see danger is using StandardTokenizerFactory for people's names? -- Regards, Ian Connor
