I working on a user-generated tagging feature. Some of the tags could be
multi-lingual, mixng languages like English, Chinese, Japanese
I'd like to add auto-complete to help users to enter the tags. And I'd want to
match in the middle of the tags as well.
For example, if a user types guit I
--- On Sat, 10/2/10, Ahmet Arslan iori...@yahoo.com wrote:
From: Ahmet Arslan iori...@yahoo.com
For example, if a user types
guit I want to suggest:
guitar
electric guitar
电动guitar
guitar英雄
And if a user types 吉他 I want to suggest:
吉他Hero
electric吉他
古典吉他
I'm
-index
everything from scratch?
The latter option just seems inefficient. Is there a better design that could
be used for this use case?
Andy
. you stored their data), or whether it's
good enough to match on question/answer separately and then
just link to a question ID in your UI to drill-down from the
database.
disclaimer: i am a solr novice - just started, so i'd see
what others think too ;)
On Oct 1, 2010, at 7:38 AM, Andy
which index do I want a document to go to?
- how do I tell solr which index do I want to query against?
- any step-by-step instruction on setting up multicore?
Thanks.
Andy
/cn/smart/SmartChineseAnalyzer.html)
However SmartChineseAnalyzer doesn't offer a separate tokenizer. It's just an
analyzer and that's it. How do I use it in Solr?
Thanks.
Andy
/
In that article KeywordTokenizerFactor is used. If I changed it to CJKTokenizer
would that work?
With an input of Apple 电脑, what would CJKTokenizer produce?
-is it Apple, 电, 脑 ?
or
- is it A, p, p, l, e, 电, 脑 ?
Any help would be greatly appreciated.
Andy
In the example solrconfig.xml that comes with Solr, the autocommit section:
autoCommit
maxDocs1/maxDocs
maxTime1000/maxTime
/autoCommit
has been commented out.
- With autoCommit commented out, does it mean that every new document indexed
to Solr is being auto-committed individually?
Thanks Mitch.
How do I do an explicit commit?
Andy
--- On Sun, 9/26/10, MitchK mitc...@web.de wrote:
From: MitchK mitc...@web.de
Subject: Re: questions about autocommit committing documents
To: solr-user@lucene.apache.org
Date: Sunday, September 26, 2010, 4:13 AM
Hi Andy,
Andy-152
--- On Thu, 9/23/10, Burton-West, Tom tburt...@umich.edu wrote:
It also splits on whitespace which causes all CJK queries
to be treated as phrase queries regardless of the CJK
tokenizer you use.
But I thought specialized analyzers like CJKAnalyzer are designed for those
languages, which
during indexing. Is there an analyzer that does that?
2) on p. 34, it mentions that the default Solr configuraton would turn L'art
into the phrase query L art but it is much more efficient to turn it into a
single token 'L art'. Which analyzer would do that?
Thanks.
Andy
Does Solr index data in a single thread or can data be indexed concurrently in
multiple threads?
Thanks
Andy
documents in different languages?
Thanks
Andy
--- On Wed, 9/22/10, Andy angelf...@yahoo.com wrote:
Does Solr index data in a single
thread or can data be indexed concurrently in multiple
threads?
Can anyone help?
Does Solr use Lucene NRT?
--- On Fri, 9/17/10, Erick Erickson erickerick...@gmail.com wrote:
From: Erick Erickson erickerick...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Friday, September 17, 2010, 1:05 PM
Near Real Time...
I'm interested in using ExternalFileField to store a field popularity that is
being updated frequently.
However ExternalFileField seems to be a pretty obscure feature. Have a few
questions:
1) Can anyone share your experience using it?
2) What is the most efficient way to update the external
, 5:16 PM
The file is completely reloaded when
you commit or optimize. There is
no incremental update available. And, yes, this could be a
scaling
problem.
How you update it is completely external to Solr.
On Sat, Aug 28, 2010 at 2:50 AM, Andy angelf...@yahoo.com
wrote:
I'm interested
...@gmail.com
Subject: Re: ExternalFileField best practices
To: solr-user@lucene.apache.org
Date: Saturday, August 28, 2010, 11:55 PM
You want the boost function bf=
parameter.
On Sat, Aug 28, 2010 at 5:32 PM, Andy angelf...@yahoo.com
wrote:
Lance,
Thanks for the response.
Can I use
I have a field popularity that is changing frequently. So I'd like to put it
in an ExternalFileField.
If I do that, can I still use popularity in a boosted query such as:
{!boost b=log(popularity)}foo
Thanks.
My documents have an expiration_datetime field that holds the expiration
datetime of the document.
I use a filter query to exclude expired documents from my query results.
Is it a good idea to periodically go through the index and remove expired
documents from it? If so what is the best way to
Is it possible to define more than 1 uniqueKey fields per document in
schema.xml?
, 2010, 5:23 PM
There can be as many as you want. Buy
you can only specify one as the
uniqueKey. That is used for Distributed Search and
deduplication.
Indexing might work better if you concatenate the different
unique
values into one field.
On Sat, Aug 21, 2010 at 3:27 AM, Andy angelf
Otis,
Thanks. In that case what does it mean that Lucene search is single threaded?
How is that different from the Solr behavior?
Andy
--- On Mon, 8/9/10, Otis Gospodnetic otis_gospodne...@yahoo.com wrote:
From: Otis Gospodnetic otis_gospodne...@yahoo.com
Subject: Re: solr single threaded
I read that Lucene search is single threaded. Does that mean Solr search is
also single threaded?
What does it mean - that there are no concurrent searches all searches are
serialized? Can Solr take advantages of multiple CPUs?
Thanks.
What if Chinese is mixed with English?
I have text that is entered by users and it could be a mix of Chinese, English,
etc.
What's the best way to handle that?
Thanks.
--- On Mon, 6/28/10, Ahmet Arslan iori...@yahoo.com wrote:
From: Ahmet Arslan iori...@yahoo.com
Subject: Re: Chinese chars
Hi,
I've seen some posts on using SOLR-1316 or Solritas for autocomplete. Wondered
what is the best solution for my use case:
1) I would like to have an hierarchical autocomplete. For example, I have a
Country dropdown list and a City textbox. A user would select a country
from the dropdown
Forgot to add, I would like to order the autocomplete suggestions for
tags/cities based on how many times they are present in the documents.
--- On Sat, 6/19/10, Andy angelf...@yahoo.com wrote:
From: Andy angelf...@yahoo.com
Subject: Non-prefix, hierarchical autocomplete? Would SOLR-1316 work
for using dismax:
{!boost b=log(popularity) defType=dismax}foo
I do this with a custom handler that I have implemented fro
my app.
Bill
On Wed, Jun 9, 2010 at 11:37 PM, Andy angelf...@yahoo.com
wrote:
I want to try out the bobo plugin for Solr, which is a
custom request
handler
I want to try out the bobo plugin for Solr, which is a custom request handler
(http://code.google.com/p/bobo-browse/wiki/SolrIntegration).
At the same time I want to use BoostQParserPlugin to boost my queries,
something like {!boost b=log(popularity)}foo
Can I use the {!boost} feature in
Yonik,
Is there any documentation where I can read more about the big core + small
core setup?
One issue for me is that I don't just add new documents. Many of the changes is
to update existing documents, such as updating the popularity score of the
documents. Would the big core + small core
#facet.method)
facet.method=fc is faster when a field has many unique terms. So how come enum,
not fc, is faster in this case?
Also why use filterCache less?
Thanks
Andy
--- On Fri, 6/4/10, Furkan Kuru furkank...@gmail.com wrote:
From: Furkan Kuru furkank...@gmail.com
Subject: Re: Faceted Search
I want to facet over a field group.
Since group is created by users, potentially there can be a huge number of
values for group.
- Would Solr be able to handle a use case like this? Or is Solr not really
appropriate for facet fields with a large number of values?
- I understand that I can set
Lucandra stores Solr index in Cassandra. What is the advantage of that compared
to regular Solr?
Anyone with experience with Lucandra Solr they can share?
--- On Sun, 4/18/10, Grant Ingersoll gsing...@apache.org wrote:
Sure, but I'm biased. ;-) Hopefully, you will find it
useful, but choose the one that best fits your needs (and
let me know if you need help assessing that.)
Thanks for the explanation Grant.
WHat is the advantage of
with documents in multiple languages?
--- On Mon, 4/19/10, MitchK mitc...@web.de wrote:
From: MitchK mitc...@web.de
Subject: Re: LucidWorks Solr
To: solr-user@lucene.apache.org
Date: Monday, April 19, 2010, 4:36 AM
Andy, I think it is important to know what a stemmer really
is.
It reduces
. 100% of the time.
Just a tip from me.
On Mon, 2010-04-19 at 00:36 -0800, MitchK wrote:
Andy, I think it is important to know what a stemmer
really is.
It reduces words to their infinitves. Those
infinitives do not refer to the
real infinitive everytime, but however
I'm setting up my Solr index to be updated every x minutes.
Does Solr cache the result of a search, and then when next time the same search
is requested, it'd recognize that the Index has not changed and therefore just
return the previous result from cache without processing the search again?
Andy,
This will help with smooth injection of your multilingual
documents into Solr (multilingual either in the sense of 1
doc containing fields in multiple languages or 1 index
containing documents in different languages):
http://sematext.com/products/multilingual-indexer/index.html
Just wanted to know if anyone has used LucidWorks Solr.
- How do you compare it to the standard Apache Solr?
- the non-blocking IO of LucidWorks Solr -- is that for networking IO or disk
IO? what are its effects?
- LucidWorks website also talked about significantly improved faceting
--- On Tue, 3/30/10, Andrzej Bialecki a...@getopt.org wrote:
From: Andrzej Bialecki a...@getopt.org
Subject: Re: SOLR-1316 How To Implement this autosuggest component ???
To: solr-user@lucene.apache.org
Date: Tuesday, March 30, 2010, 9:59 AM
On 2010-03-30 15:42, Robert Muir
wrote:
On
Reading through this thread and SOLR-1316, there seems to be a lot of different
ways to implement auto-complete in Solr. I've seen the mentions of:
EdgeNGrams
TermsComponent
Faceting
TST
Patricia Tries
RadixTree
DAWG
Which algorthm does SOLR-1316 implement? TST is one. There are others
...@mitre.org
Subject: Re: How many facet values are too many?
To: solr-user@lucene.apache.org solr-user@lucene.apache.org
Date: Friday, March 19, 2010, 9:07 AM
On Mar 18, 2010, at 10:53 PM, Andy wrote:
My understanding is that too many facet values will decrease performance
How many is too many
My understanding is that too many facet values will decrease performance
How many is too many? Are there any rules of thumb for this?
2 related questions:
- I expect a facet field to have many values (values are user generated), any
thing I can do to minimize the performance impact?
- Any way
There's a field A I want to facet on.
Some documents have no value for field A. So they wouldn't show up in the
list of facet value options.
I want to find a way to let users to find those documents. One way is to make
Null an option the users can choose, something like:
value1 (4558)
value2
on null value
To: solr-user@lucene.apache.org
Date: Thursday, March 4, 2010, 7:13 PM
This query will find them: *:* AND -A:[* TO *]
The '*:* AND' is to get around a weird quirk of Lucene. Minus field
range star TO star is the trick.
On Thu, Mar 4, 2010 at 3:06 PM, Andy angelf...@yahoo.com wrote
Yes. Thank you.
--- On Thu, 3/4/10, Koji Sekiguchi k...@r.email.ne.jp wrote:
From: Koji Sekiguchi k...@r.email.ne.jp
Subject: Re: facet on null value
To: solr-user@lucene.apache.org
Date: Thursday, March 4, 2010, 7:21 PM
Andy wrote:
There's a field A I want to facet on.
Some documents have
What would the response look like with this query?
Can you give an example?
--- On Thu, 3/4/10, Chris Hostetter hossman_luc...@fucit.org wrote:
From: Chris Hostetter hossman_luc...@fucit.org
Subject: Re: facet on null value
To: solr-user@lucene.apache.org
Date: Thursday, March 4, 2010, 8:40 PM
. You get all
of the 0-valued facets anyway.
What exactly is facet.missing supposed to do with this query?
On Thu, Mar 4, 2010 at 6:39 PM, Andy angelf...@yahoo.com wrote:
What would the response look like with this query?
Can you give an example?
--- On Thu, 3/4/10, Chris Hostetter hossman_luc
be as simple as:
facet=onfacet.field={!key=Location}_loc_hier_cityareafq=_loc_hier_city:somecityid
Cheers,
Geert-Jan
2010/3/3 Andy angelf...@yahoo.com
Thanks. I didn't know about the {!key=Location} trick.
Thanks everyone for your help. From what I could gather, there're 3
approaches:
1) SOLR
I have a facet field whose values are created by users. So potentially there
could be a very large number of values. is that going to be a problem
performance-wise?
A few more questions to help me understand how facet works:
- after the filter cache warmed up, will the (if any) performance
hierarchical facets, but
would be surprised if it has a (major) speed increase to this method)
- flexible (you get multiple hierarchies: country -- state -- city and
country -- city)
cons:
- a little more application logic
Hope that helps,
Geert-Jan
2010/3/2 Andy angelf...@yahoo.com
I read that a simple way to implement hierarchical facet is to concatenate
strings with a separator. Something like level1level2level3 with as the
separator.
A problem with this approach is that the number of facet values will greatly
increase.
For example I have a facet Location with the
Oops. Sorry about that.
I'll start a fresh one.
--- On Mon, 3/1/10, Chris Hostetter hossman_luc...@fucit.org wrote:
From: Chris Hostetter hossman_luc...@fucit.org
Subject: Re: Implementing hierarchical facet
To: solr-user@lucene.apache.org
Date: Monday, March 1, 2010, 11:36 PM
: Subject:
(repost with a fresh email)
I read that a simple way to implement hierarchical facet is to
concatenate strings with a separator. Something like
level1level2level3 with as the separator.
A problem with this approach is that the number of facet values will greatly
increase.
For
example I have a
The Solr documentation feels more like a reference guide detailing all the
API's. It's great for more advanced users, but as a beginner I often feel lost
reading the doc.
It would be really helpful to have a more step-by-step, tutorial approach in
the doc showing how to do things with tips
1) Built-in hierarchical faceting
Right now there're 2 patches, SOLR-64 and SOLR-792. SOLR-64 seems to be slated
for 1.5 release but according to the wiki seems to have poor performance.
SOLR-792 has better performance according to the wiki but it's unclear if it'll
ever be part of the Solr
This read more like a PR release or product brochure for jetty than anything
else.
Then I poked around the website and realized why: it was written by the creator
of Jetty, and is hosted on the website of a company with the slogan The Java
Experts behind Jetty
--- On Wed, 2/17/10,
My documents have a field expiration that is the expiration date of that doc.
I want to give a boost to all documents that haven't expired. I still want to
have expired documents returned, but unexpired documents should be given
priority.
Ideally the boost amount for all unexpired documents
within that
date range. Check out the 'boost query' feature of dismax.
http://www.lucidimagination.com/search/document/CDRG_ch07_7.4.2.9
It's also possible with the standard query parser but a
pain in the neck:
(value)^2 OR (NOT value)
On Thu, Jan 28, 2010 at 6:58 PM, Andy angelf
I'd like to provide a hierarchical faceting functionality.
An example would be location drill down such as USA - New York - New York
City - SoHo
The number of levels can be arbitrary. One way to handle this could be to use a
special character as separator, store values such as USA|New York|New
Solr comes with Jetty. But I noticed that Lucid Solr switched to Tomcat. What
is the reason for that?
Is there a recommended servlet container for Solr?
Is it possible to set up Solr such that when there's no query (client would
send in *:* for q), Solr would sort results (basically all the documents)
by date or some other criterion.
And when there is a query other than *:*, I'd want the results to be ranked
by score.
Is that something that's
- Nutch
- Original Message
From: Andy angelf...@yahoo.com
To: solr-user@lucene.apache.org
Sent: Fri, January 15, 2010 3:47:26 PM
Subject: only use sorting when there's no q is
*:*?
Is it possible to set up Solr such that when there's
no query (client would send
if this was answered.
Yes, you can set the default params/values for a request handler in the
solrconfig.xml .
Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
- Original Message
From: Andy angelf...@yahoo.com
To: solr-user@lucene.apache.org
Sent: Mon, January 4
, in the config file.
q.alt is typically *:*, it's the fall back query when no q is provided.
though, in thinking about it, q.alt would work here, but i'd use q personally.
On Jan 6, 2010, at 9:45 PM, Andy wrote:
Let me make sure I understand you.
I'd get my regular query from haystack as qq=foo
, Erik Hatcher erik.hatc...@gmail.com wrote:
From: Erik Hatcher erik.hatc...@gmail.com
Subject: Re: DisMaxRequestHandler bf configuration
To: solr-user@lucene.apache.org
Date: Thursday, January 7, 2010, 9:57 PM
On Jan 7, 2010, at 9:51 PM, Andy wrote:
Thanks.
Can I use the standard request
, Yonik Seeley yo...@lucidimagination.com wrote:
From: Yonik Seeley yo...@lucidimagination.com
Subject: Re: DisMaxRequestHandler bf configuration
To: solr-user@lucene.apache.org
Date: Wednesday, January 6, 2010, 7:09 PM
On Wed, Jan 6, 2010 at 2:43 AM, Andy angelf...@yahoo.com wrote:
I'd like
.
--- On Wed, 1/6/10, Yonik Seeley yo...@lucidimagination.com wrote:
From: Yonik Seeley yo...@lucidimagination.com
Subject: Re: DisMaxRequestHandler bf configuration
To: solr-user@lucene.apache.org
Date: Wednesday, January 6, 2010, 7:48 PM
On Wed, Jan 6, 2010 at 7:43 PM, Andy angelf...@yahoo.com
/6/10, Yonik Seeley yo...@lucidimagination.com wrote:
From: Yonik Seeley yo...@lucidimagination.com
Subject: Re: DisMaxRequestHandler bf configuration
To: solr-user@lucene.apache.org
Date: Wednesday, January 6, 2010, 8:42 PM
On Wed, Jan 6, 2010 at 8:24 PM, Andy angelf...@yahoo.com wrote:
I meant
I'd like to boost every query using {!boost b=log(popularity)}. But I'd rather
not have to prepend that to every query. It'd be much cleaner for me to
configure Solr to use that as default.
My plan is to make DisMaxRequestHandler the default handler and add the
following to solrconfig.xml:
Thank you Ahmet.
Is there any way I can configure Solr to always use {!boost b=log(popularity)}
as the default for all queries?
I'm using Solr through django-haystack, so all the Solr queries are actually
generated by haystack. It'd be much cleaner if I could configure Solr to always
use
Is there any way to modify result ranking using an integer field?
I have documents that have an integer field popularity.
I want to rank results by a combination of normal fulltext search
relevance and popularity. It's kinda like search in digg - result
ranking is based on the search
Thanks Ahmet.
Do I need to do anything to enable BoostQParserPlugin in Solr, or is it already
enabled?
--- On Sun, 1/3/10, Ahmet Arslan iori...@yahoo.com wrote:
From: Ahmet Arslan iori...@yahoo.com
Subject: Re: Any way to modify result ranking using an integer field?
To:
What I meant was that is there any way to make {!boost b=log(popularity)} the
default query type so that every query will be using it.
From: Andy angelf...@yahoo.com
Subject: Re: Any way to modify result ranking using an integer field?
To: solr-user@lucene.apache.org
Date: Monday, January 4
I have documents that represent Companies. Each company has a field Internet
which is a boolean field, True means the company is an Internet company.
There's also another field location which is the city the company is located
in, eg. Austin or Houston.
A company can be both an Internet
as the application layer, hence as a workaround if
there are any ways that I can get the xhtml result, maybe I can stored it
somewhere else outside of Solr.
Any advice on this will be highly appreciated.
Many Thanks Kind Regards
Andy
give good advice on possible solution concepts.
I'm thinking that rather than using file-system tricks, perhaps there
might be a simple means to switch the file system location that the
index readers use within SOLR.
thanks
Andy
and one
tokenized field using class TextField. Very easy to feed into both using
copyField.
Your query would need to use the different field name instead of an exact
operator like with some commercial search engines.
hope that helps,
Andy
Jörg Kiegeland wrote:
Normally I do substring-queries
.
Thans
Andy.
--
View this message in context:
http://www.nabble.com/Slow-response-times-using-*%3A*-tp15206563p15206563.html
Sent from the Solr - User mailing list archive at Nabble.com.
of memory if the number of unique terms
are large.
On Jan 31, 2008 9:13 PM, Andy Blower [EMAIL PROTECTED] wrote:
I'm evaluating SOLR/Lucene for our needs and currently looking at
performance
since 99% of the functionality we're looking for is provided. The index
contains 18.4 Million records
Yonik Seeley wrote:
*:* maps to MatchAllDocsQuery, which for each document needs to check
if it's deleted (that's a synchronized call, and can be a bottleneck).
Why does this need to check if documents are deleted if normal queries
don't? Is there any way of disabling this since I can be
201 - 281 of 281 matches
Mail list logo
