Re: K-Stemmer for Solr 3.1
Thanks. Ill have to create a Jira account to vote i guess. We are already using KStemmer in 1.4.2 production and I would like to upgrade to 3.1. In the meantime, what is another stemmer I could use out of the box that would be have similar to KStemmer? Thanks On 5/28/11 10:02 AM, Steven A Rowe wrote: Hi Mark, Yonik Seeley indicated on LUCENE-152 that he is considering contributing Lucid's KStemmer version to Lucene: https://issues.apache.org/jira/browse/LUCENE-152?focusedCommentId=13035647page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13035647 You can vote on the issue to communicate your interest. Steve -Original Message- From: Mark [mailto:static.void@gmail.com] Sent: Friday, May 27, 2011 7:31 PM To: solr-user@lucene.apache.org Subject: Re: K-Stemmer for Solr 3.1 Where can one find the KStemmer source for 4.0? On 5/12/11 11:28 PM, Bernd Fehling wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd
RE: K-Stemmer for Solr 3.1
Hi Mark, Yonik Seeley indicated on LUCENE-152 that he is considering contributing Lucid's KStemmer version to Lucene: https://issues.apache.org/jira/browse/LUCENE-152?focusedCommentId=13035647page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13035647 You can vote on the issue to communicate your interest. Steve -Original Message- From: Mark [mailto:static.void@gmail.com] Sent: Friday, May 27, 2011 7:31 PM To: solr-user@lucene.apache.org Subject: Re: K-Stemmer for Solr 3.1 Where can one find the KStemmer source for 4.0? On 5/12/11 11:28 PM, Bernd Fehling wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd
Re: K-Stemmer for Solr 3.1
Where can one find the KStemmer source for 4.0? On 5/12/11 11:28 PM, Bernd Fehling wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd
Re: K-Stemmer for Solr 3.1
I see KStem being mentioned lately. It's been 5+ years since I looked at the original KStem stuff, but I recall there being a license issue with the *original* KStem. I think it was under some flavour of GPL and that was the reason why we didn't include it in Lucene/Solr back then. I say this now because I saw people said KStem was released under BSD license, which doesn't match what I saw 5+ years ago. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message From: Smiley, David W. dsmi...@mitre.org To: solr-user@lucene.apache.org solr-user@lucene.apache.org Sent: Mon, May 16, 2011 5:33:00 PM Subject: Re: K-Stemmer for Solr 3.1 Lucid's KStemmer is LGPL and the Solr committers have shown that they don't want LGPL libraries shipping with Solr. If you are intent on releasing your changes, I suggest attaching both the modified source and the compiled jar onto Solr's k-stemmer wiki page; and of course say that it's LGPL licensed. ~ David Smiley On May 16, 2011, at 2:24 AM, Bernd Fehling wrote: I don't know if it is allowed to modify Lucid code and add it to jira. If someone from Lucid would give me the permission and the Solr developers have nothing against it I won't mind adding the Lucid KStemmer to jira for Solr 3.x and 4.x. There are several Lucid KStemmer users which I can see from the many requests which I got. Also the Lucid KStemmer is faster than the standard KStemmer. Bernd Am 16.05.2011 06:33, schrieb Bill Bell: Did you upload the code to Jira? On 5/13/11 12:28 AM, Bernd Fehlingbernd.fehl...@uni-bielefeld.de wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd -- * Bernd Fehling Universitätsbibliothek Bielefeld Dipl.-Inform. (FH) Universitätsstr. 25 Tel. +49 521 106-4060Fax. +49 521 106-4052 bernd.fehl...@uni-bielefeld.de 33615 Bielefeld BASE - Bielefeld Academic Search Engine - www.base-search.net *
Re: K-Stemmer for Solr 3.1
Hm, maybe I was wrong. I don't see any mention of *GPL on KStem download page. I only see http://ciir.cs.umass.edu/downloads/agreements/general.html. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message From: Otis Gospodnetic otis_gospodne...@yahoo.com To: solr-user@lucene.apache.org Sent: Wed, May 18, 2011 11:35:32 PM Subject: Re: K-Stemmer for Solr 3.1 I see KStem being mentioned lately. It's been 5+ years since I looked at the original KStem stuff, but I recall there being a license issue with the *original* KStem. I think it was under some flavour of GPL and that was the reason why we didn't include it in Lucene/Solr back then. I say this now because I saw people said KStem was released under BSD license, which doesn't match what I saw 5+ years ago. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message From: Smiley, David W. dsmi...@mitre.org To: solr-user@lucene.apache.org solr-user@lucene.apache.org Sent: Mon, May 16, 2011 5:33:00 PM Subject: Re: K-Stemmer for Solr 3.1 Lucid's KStemmer is LGPL and the Solr committers have shown that they don't want LGPL libraries shipping with Solr. If you are intent on releasing your changes, I suggest attaching both the modified source and the compiled jar onto Solr's k-stemmer wiki page; and of course say that it's LGPL licensed. ~ David Smiley On May 16, 2011, at 2:24 AM, Bernd Fehling wrote: I don't know if it is allowed to modify Lucid code and add it to jira. If someone from Lucid would give me the permission and the Solr developers have nothing against it I won't mind adding the Lucid KStemmer to jira for Solr 3.x and 4.x. There are several Lucid KStemmer users which I can see from the many requests which I got. Also the Lucid KStemmer is faster than the standard KStemmer. Bernd Am 16.05.2011 06:33, schrieb Bill Bell: Did you upload the code to Jira? On 5/13/11 12:28 AM, Bernd Fehlingbernd.fehl...@uni-bielefeld.de wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd -- * Bernd Fehling Universitätsbibliothek Bielefeld Dipl.-Inform. (FH) Universitätsstr. 25 Tel. +49 521 106-4060 Fax. +49 521 106-4052 bernd.fehl...@uni-bielefeld.de 33615 Bielefeld BASE - Bielefeld Academic Search Engine - www.base-search.net *
Re: K-Stemmer for Solr 3.1
Hi Otis, conclusion, if we check that the license agreement is included in all source files and as a seperate license file then we are clear about KStem itself. What about the modifications from Lucid, do you know if they publish under GPL? Bernd - BASE - Bielefeld Academic Search Engine http://www.base-search.net/ Am 19.05.2011 05:39, schrieb Otis Gospodnetic: Hm, maybe I was wrong. I don't see any mention of *GPL on KStem download page. I only see http://ciir.cs.umass.edu/downloads/agreements/general.html. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message From: Otis Gospodneticotis_gospodne...@yahoo.com To: solr-user@lucene.apache.org Sent: Wed, May 18, 2011 11:35:32 PM Subject: Re: K-Stemmer for Solr 3.1 I see KStem being mentioned lately. It's been 5+ years since I looked at the original KStem stuff, but I recall there being a license issue with the *original* KStem. I think it was under some flavour of GPL and that was the reason why we didn't include it in Lucene/Solr back then. I say this now because I saw people said KStem was released under BSD license, which doesn't match what I saw 5+ years ago. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message From: Smiley, David W.dsmi...@mitre.org To: solr-user@lucene.apache.orgsolr-user@lucene.apache.org Sent: Mon, May 16, 2011 5:33:00 PM Subject: Re: K-Stemmer for Solr 3.1 Lucid's KStemmer is LGPL and the Solr committers have shown that they don't want LGPL libraries shipping with Solr. If you are intent on releasing your changes, I suggest attaching both the modified source and the compiled jar onto Solr's k-stemmer wiki page; and of course say that it's LGPL licensed. ~ David Smiley On May 16, 2011, at 2:24 AM, Bernd Fehling wrote: I don't know if it is allowed to modify Lucid code and add it to jira. If someone from Lucid would give me the permission and the Solr developers have nothing against it I won't mind adding the Lucid KStemmer to jira for Solr 3.x and 4.x. There are several Lucid KStemmer users which I can see from the many requests which I got. Also the Lucid KStemmer is faster than the standard KStemmer. Bernd Am 16.05.2011 06:33, schrieb Bill Bell: Did you upload the code to Jira? On 5/13/11 12:28 AM, Bernd Fehlingbernd.fehl...@uni-bielefeld.de wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd -- * Bernd Fehling Universitätsbibliothek Bielefeld Dipl.-Inform. (FH) Universitätsstr. 25 Tel. +49 521 106-4060 Fax. +49 521 106-4052 bernd.fehl...@uni-bielefeld.de 33615 Bielefeld BASE - Bielefeld Academic Search Engine - www.base-search.net * -- * Bernd FehlingUniversitätsbibliothek Bielefeld Dipl.-Inform. (FH)Universitätsstr. 25 Tel. +49 521 106-4060 Fax. +49 521 106-4052 bernd.fehl...@uni-bielefeld.de33615 Bielefeld BASE - Bielefeld Academic Search Engine - www.base-search.net *
Re: K-Stemmer for Solr 3.1
I don't know if it is allowed to modify Lucid code and add it to jira. If someone from Lucid would give me the permission and the Solr developers have nothing against it I won't mind adding the Lucid KStemmer to jira for Solr 3.x and 4.x. There are several Lucid KStemmer users which I can see from the many requests which I got. Also the Lucid KStemmer is faster than the standard KStemmer. Bernd Am 16.05.2011 06:33, schrieb Bill Bell: Did you upload the code to Jira? On 5/13/11 12:28 AM, Bernd Fehlingbernd.fehl...@uni-bielefeld.de wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd -- * Bernd FehlingUniversitätsbibliothek Bielefeld Dipl.-Inform. (FH)Universitätsstr. 25 Tel. +49 521 106-4060 Fax. +49 521 106-4052 bernd.fehl...@uni-bielefeld.de33615 Bielefeld BASE - Bielefeld Academic Search Engine - www.base-search.net *
Re: K-Stemmer for Solr 3.1
Lucid's KStemmer is LGPL and the Solr committers have shown that they don't want LGPL libraries shipping with Solr. If you are intent on releasing your changes, I suggest attaching both the modified source and the compiled jar onto Solr's k-stemmer wiki page; and of course say that it's LGPL licensed. ~ David Smiley On May 16, 2011, at 2:24 AM, Bernd Fehling wrote: I don't know if it is allowed to modify Lucid code and add it to jira. If someone from Lucid would give me the permission and the Solr developers have nothing against it I won't mind adding the Lucid KStemmer to jira for Solr 3.x and 4.x. There are several Lucid KStemmer users which I can see from the many requests which I got. Also the Lucid KStemmer is faster than the standard KStemmer. Bernd Am 16.05.2011 06:33, schrieb Bill Bell: Did you upload the code to Jira? On 5/13/11 12:28 AM, Bernd Fehlingbernd.fehl...@uni-bielefeld.de wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd -- * Bernd FehlingUniversitätsbibliothek Bielefeld Dipl.-Inform. (FH)Universitätsstr. 25 Tel. +49 521 106-4060 Fax. +49 521 106-4052 bernd.fehl...@uni-bielefeld.de33615 Bielefeld BASE - Bielefeld Academic Search Engine - www.base-search.net *
RE: K-Stemmer for Solr 3.1
On 5/16/2011 at 5:33 PM, David W. Smiley wrote: Lucid's KStemmer is LGPL and the Solr committers have shown that they don't want LGPL libraries shipping with Solr. If you are intent on releasing your changes, I suggest attaching both the modified source and the compiled jar onto Solr's k-stemmer wiki page; and of course say that it's LGPL licensed. AFAICT, all Apache MoinMoin wikis (at least Lucene's and Solr's) have disabled attachments - you can't retrieve existing attachments, and you can't create new ones. (Spam, apparently, was the impetus for this change.) Steve
Re: K-Stemmer for Solr 3.1
On Mon, May 16, 2011 at 5:33 PM, Smiley, David W. dsmi...@mitre.org wrote: Lucid's KStemmer is LGPL and the Solr committers have shown that they don't want LGPL libraries shipping with Solr. If you are intent on releasing your changes, I suggest attaching both the modified source and the compiled jar onto Solr's k-stemmer wiki page; and of course say that it's LGPL licensed. ~ David Smiley Hi David, I don't know much about this stemmer but the original implementation is BSD-licensed (http://ciir.cs.umass.edu/cgi-bin/downloads/downloads.cgi)
Re: K-Stemmer for Solr 3.1
Did you upload the code to Jira? On 5/13/11 12:28 AM, Bernd Fehling bernd.fehl...@uni-bielefeld.de wrote: I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd
Re: K-Stemmer for Solr 3.1
I backported a Lucid KStemmer version from solr 4.0 which I found somewhere. Just changed from import org.apache.lucene.analysis.util.CharArraySet; // solr4.0 to import org.apache.lucene.analysis.CharArraySet; // solr3.1 Bernd Am 12.05.2011 16:32, schrieb Mark: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd
Re: K-Stemmer for Solr 3.1
Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd
Re: K-Stemmer for Solr 3.1
java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z Would you mind explaining your modifications? Thanks On 5/11/11 11:14 PM, Bernd Fehling wrote: Am 12.05.2011 02:05, schrieb Mark: It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks Lucid KStemmer works nice with Solr3.1 after some minor mods to KStemFilter.java and KStemFilterFactory.java. What problems do you have? Bernd
K-Stemmer for Solr 3.1
It appears that the older version of the Lucid Works KStemmer is incompatible with Solr 3.1. Has anyone been able to get this to work? If not, what are you using as an alternative? Thanks
