Thanks Eric.
Here's the part which I'm not able to understand. I've for e.g. Source A, B,
C and D in index. Each source contains n number of documents. Now, out of
these, a bunch of documents in A and B are tagged with MediaType. I took the
following steps:
1. Delete all documents tagged with
I didn't use the REST API, instead updated the schema manually.
Can you be specific on removing the data directory content ? I certainly
don't want to wipe out the index. I've four Solr instances, 2 shards with a
replica each. Are you suggesting clearing the index and re-indexing from
scratch ?
results back if I
don't use result grouping.
Any pointers will be appreciated.
- Thanks,
Shamik
,
Shamik
Anyone ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Issue-with-German-search-tp4206104p4206306.html
Sent from the Solr - User mailing list archive at Nabble.com.
Thanks Doug. I'm using eDismax
Here's my Solr query :
http://localhost:8983/solr/testhandlerdeu?debugQuery=trueq=title_deu:Software%20und%20Downloads
Here's my request handler.
requestHandler name=/testhandlerdeu class=solr.SearchHandler
lst name=defaults
Thanks a ton Doug, I should have figured this out, pretty stupid of me.
Appreciate your help.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Issue-with-German-search-tp4206104p4206357.html
Sent from the Solr - User mailing list archive at Nabble.com.
Sie, wie Sie eine Vorgängerversion Ihrer
Autodesk-Software erhaltenSprachoptionenHerunterladen der lizenzierten
Software in einer anderen Sprache oder Erhalten eines Sprachpakets.
/field
field name=authorBob/field
/doc
Any pointers will be appreciated.
-Thanks,
Shamik
You should look at CollapsinQParserPlugin. It's much faster compared to a
Grouping query.
https://wiki.apache.org/solr/CollapsingQParserPlugin
It has a limitation though, check the following JIRA if it might affect your
use-case.
https://issues.apache.org/jira/browse/SOLR-6143
--
View this
Ok, I figured the steps in case someone needs a reference. It required both
zkcli and RELOAD to update the changes.
1. Use zkcli to load the changes. I ran it from the node which used the
bootstrapping.
sh zkcli.sh -cmd upconfig -zkhost zoohost1:2181 -confname myconf -solrhome
Thanks Shawn for the pointer, really appreciate it.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Unable-to-update-config-file-using-zkcli-or-RELOAD-tp4197376p4197494.html
Sent from the Solr - User mailing list archive at Nabble.com.
/collection1/newhandler
I got a 404. Not sure what I'm doing wrong. Do I need to run the zkcli
script on each node? I'm using Solr 5.0.
Regards,
Shamik
will be appreciated.
Regards,
Shamik
Thanks for your reply Eric.
In my case, I've 14 languages, out of which 50% of the documents belong to
English. German and CHS will probably constitute another 25%. I'm not using
copyfield, rather, each language has it's dedicated field such as title_enu,
text_enu, title_ger,text_ger, etc. Since
will be appreciated.
Regards,
Shamik
'/strint
name=code400/int/lst
What am I missing here ? I'm using Solr 5.0 version.
Any pointers will be appreciated.
Regards,
Shamik
the error on any group by query -- SolrDispatchFilter null:java.lang.
IllegalStateException: unexpected docvalues type NONE for field 'ADSKDedup'
(expected=SORTED). Use UninvertingReader or dex with docvalues.
Does this mean that I need to re-index documents to get over this error ?
Regards,
Shamik
Wow, optimize worked like a charm. This really addressed the docvalues
issue. A follow-up question, is it recommended to run optimize in a
Production Solr index ? Also, in a Sorl cloud mode, do we need to run
optimize on each instance / each shard / any instance ?
Appreciate your help Alex.
--
Well, I think I've narrowed down the issue. The error is happening when I'm
trying to do a rolling update from Solr 4.7 (which is our current version)
to 5.0 . I'm able to re-produce this couple of times. If I do a fresh index
on a 5.0, it works. Not sure if there's any other way to mitigate it.
Looks like it's happening for any field which is using docvalues.
java.lang.IllegalStateException: unexpected docvalues type NONE for field
'title_sort' (expected=SORTED). Use UninvertingReader or index with
docvalues.
Any idea ?
--
View this message in context:
Thanks for your reply. Initially, I was under the impression that the issue
is related to grouping as group queries were failing. Later, when I looked
further, I found that it's happening for any field for which the docvalue
has turned on. The second example I took was from another field. Here's a
The other way you can do that is to specify the startup parameters in
solr.in.sh.
Example :
SOLR_MODE=solrcloud
ZK_HOST=zoohost1:2181,zoohost2:2181,zoohost3:2181
SOLR_PORT=4567
You can simply start solr by running ./solr start
--
View this message in context:
Joel,
To give you some context, we are running queries against 6 million
documents in a Solr cloud environment. The grouping is done to de-duplicate
content based on an unique field. Unfortunately, due to some requirement
constraint, the only way for us to run the de-duplication is during query
devic
I'm in Solr 4.7, so not sure if enablePositionIncrements=true is making
any difference.
Any feedback will be appreciated.
Thanks,
Shamik
Well, I somehow made it work by using CommonGramsFilterFactory.
filter class=solr.CommonGramsFilterFactory words=stopwords.txt
ignoreCase=true/
Just wondering if it's the right approach ?
--
View this message in context:
I'm missing here, any pointers will be appreciated.
Thanks,
Shamik
Thanks Hoss for clearing up my doubt. I was confused with the ordering. So I
guess, the first field is always the primary sort field followed by
secondary.
Thanks again.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Issue-with-Solr-multiple-sort-tp4181056p4181062.html
This one worked.
if(termfreq(Source,'A'),sum(Likes,3),if(termfreq(Source,'B'),sum(Likes,3),0))
--
View this message in context:
http://lucene.472066.n3.nabble.com/Conditions-in-Boost-function-query-tp4179687p4179906.html
Sent from the Solr - User mailing list archive at Nabble.com.
wondering if anyone have tried DocValues for Grouping and saw
any improvements ?
-Thanks,
Shamik
.
For e.g.
if Source=A or B or C, then sum(Likes,4)
if Source=D then sum(Likes,3)
if Source=E the sum(Likes,2).
Is it possible to do this using a function ?
Any pointers will be appreciated.
Regards,
Shamik
Thanks Eric, I did take a look at the if condition earlier, but not sure
how that can be used for multiple conditions. It works for a single
condition :
if(termfreq(Source2,'A'),sum(Likes,3),0)
But for multiple, I'm struggling to find the right syntax. I tried using OR
in conjunction but hasn't
Anyone ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Have-anyone-used-Automatic-Phrase-Tokenization-AutoPhrasingTokenFilterFactory-tp4173808p4174069.html
Sent from the Solr - User mailing list archive at Nabble.com.
Ted,
Here's the query I'm using and the debug info. It's still returning all 5
results back as if it's simply looking for either of the term with q.op set
as OR (default).
http://localhost:8983/solr/autophrase?q=text:seat+cushionswt=xmldebugQuery=true
Debug
lst name=debug
str
Jim,
Thanks for your response. I've tried including
AutoPhrasingTokenFilterFactory as part of the query analyzer, but didn't
make any difference.
fieldType name=text_autophrase class=solr.TextField
positionIncrementGap=100
analyzer type=index
tokenizer
Ted,
Thanks a lot, I had gone through your blogs but the white space issue
slipped out of my mind. replaceWhitespaceWith addressed the issue. I think
it's a great filter to have, surely takes care of an important use case.
Appreciate your help.
-Shamik
--
View this message in context
.
-Thanks
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/Have-anyone-used-Automatic-Phrase-Tokenization-AutoPhrasingTokenFilterFactory-tp4173808.html
Sent from the Solr - User mailing list archive at Nabble.com.
com.lucidworks.analysis.AutoPhrasingQParserPlugin supports the edismax
features which is my default parser.
I'll appreciate if anyone provide their feedback.
-Thanks
Shamik
the value from solrconfig and defaulting it
to em.
Can someone provide any pointer ? I'm using Solr 4.7.
Thanks,
Shamik
Looks like this has to do with the selection of fast vector and
breakIterator as boundary scanner. I'm using them to make sure that the
highlighted snippet starts from the beginning of a sentence and not from
the middle.
str name=hl.usePhraseHighlighterfalse/str
str
Found the issue, to use FastVectorHighlighter, the pre and post tag syntax
are different
str name=hl.tag.pre/str
str name=hl.tag.post/str
This worked out as expected.
--
View this message in context:
Thanks a lot Jack, it makes total sense. I check the config and default q.op
was set to OR, which was influencing the query.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Boost-Query-bq-syntax-usage-tp4161989p4162169.html
Sent from the Solr - User mailing list archive at
Hi,
I'm little confused with the right syntax of defining boost queries. If I
use them in the following way:
http://localhost:8983/solr/testhandler?q=Application+Managerbq=(Source2:sfdc^6
Source2:downloads^5 Source2:topics^3)debugQuery=true
it gets translated to --
arr
Hi,
I'm little confused with the right syntax of defining boost queries. If I
use them in the following way:
http://localhost:8983/solr/testhandler?q=Application+Managerbq=(Source2:sfdc^6
Source2:downloads^5 Source2:topics^3)debugQuery=true
it gets translated to --
arr
Thanks a lot Jack, makes sense. Just curios, if we used the following bq
entry in solrconfig xml
str name=bqSource2:sfdc^6 Source2:downloads^5 Source2:topics^3/str
will it always be treated as an AND query ? Some of local results suggests
otherwise.
--
View this message in context:
in the lines boost ProductLine^5 if Author:Tom.
I've been using a similar filtering in appends section, but not sure how
to do it in qf or whether it's possible.
lst name=appends
str name=fqAuthor:(Tom +Solution:yes) /str
/lst
Any pointers will be appreciated.
Thanks,
Shamik
Thanks Markus. Well, I tried using a conditional if-else function, but it
doesn't seem to work for boosting field. What I'm trying to do is boost
ProductLine field by 5, if the result documents contain Author = 'Tom'.
--
View this message in context:
Thanks Markus, let me play around with the functions and see if I can achieve
the results.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-query-field-qf-conditional-boost-tp4161783p4161803.html
Sent from the Solr - User mailing list archive at Nabble.com.
query time.
Appreciate your feedback.
-Thanks,
Shamik
Thanks Jack for your reply ... I'm sorry but I'm not too clear on the
solution you proposed. Can you please provide a sample on what you suggested
?
--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-query-certain-fields-filtered-by-a-condition-tp4161815p4161827.html
with CzechStemFilterFactory. I know there'll be trade-offs with various
stemmers, but not sure which one fits the bill. Being an alien to Czech
language doesn't help the cause either.
Thanks,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/Czech-stemmer-tp4157675p4158301
=solr.CzechStemFilterFactory/
/analyzer
/fieldType
Any pointers will be appreciated.
- Thanks,
Shamik
I've noticed similar behavior with our Solr cloud cluster for a while, it's
random though. We've 2 shards with 3 replicas each. At times, I've observed
that the same query on refresh will fetch different results (numFound) as
well as the content. The only way to mitigate is to refresh the index
Are there any plans to release this feature anytime soon ? I think this is
pretty important as a lot of search use case are dependent on the facet
count being returned by the search result. This issue renders renders the
CollapsingQParserPlugin pretty much unusable. I'm now reverting back to the
Yes it does and pretty straight forward.
Refer to following url :
http://heliosearch.org/solr/atomic-updates/
http://www.mumuio.com/solrj-4-0-0-alpha-atomic-updates/
--
View this message in context:
Thanks for the pointer Eric. You are right, I forgot to include IJK under
AB. Also, facet field names are different. Unfortunately, I'm using
Solrcloud and facet pivot doesn't seem to work in a distributed mode. I'll
get back some result if I use distrib=false, but then it's not the right
data.
Anyone ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/MLT-weird-behaviour-in-Solrcloud-tp4145066p4145502.html
Sent from the Solr - User mailing list archive at Nabble.com.
Facet:XYZ
3. Doc C -- Facet:AB,CD
Facet:IJK, XYZ
Now, I want the result organized as :
AB
MNO,XYZ
CD
IJK,XYZ
Is there a way to do this ?
Thanks,
Shamik
.
Is this a known issue with solrcloud ? Any pointer will be appreciated.
Thanks,
Shamik
Sorry, that's a typo when I copied the mlt definition from my solrconfig, but
there's comma in my test environment. It's not the issue.
--
View this message in context:
http://lucene.472066.n3.nabble.com/MLT-weird-behaviour-in-Solrcloud-tp4145066p4145145.html
Sent from the Solr - User mailing
, as well as, boosting
on source. What I'm trying to see is if there's a way to change the the
weights based on Source.E.g. for source Blog, I would like to have the
following boost text^3 title^2 while for source Videos , I prefer
text^2 title^3.
Any pointers will be appreciated.
Thanks,
Shamik
Thanks Ahmet, I'll give it a shot.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Can-we-do-conditional-boosting-using-edismax-tp4141131p4141268.html
Sent from the Solr - User mailing list archive at Nabble.com.
will be appreciated.
Thanks,
Shamik
Turned out to be a weird exception. Apparently, the comments in the
stopwords_fr.txt disrupts the stop filter factory. After I stripped off the
comments, it worked as expected.
Referred to this thread :
I found the issue. It had to do with edismax qf entry in request handler. I
had the following entry :
str name=qfname_fra^1.2 title_fra^10.0 description_fra^5.0
author^1/str
Except for author, all other fields are of type adsktext_fra, while author
was of the type text_general, which uses
i.e. app level. This is required because the usage of the router as 2
or 3 level isn’t implicit. Specifying ’8′ bits for the component highlights
the use of ’3′ level router*.
Any feedback will be much appreciated.
Thanks,
Shamik
Awesome, thanks a lot Anshum, makes total sense now. Appreciate your help.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Question-on-3-level-composite-id-routing-tp4137044p4137071.html
Sent from the Solr - User mailing list archive at Nabble.com.
Thanks Nicole. Leveraging dynamic field definitions is a great idea. Probably
work for me as I've a bunch of fields which are indexed as String. Just
curious about the sharding, are you using Solr Cloud. I thought of taking
the dedicated shard / core route , but then, as using a composite key (for
pointer on this will be highly appreciated.
Thanks,
Shamik
/ parsedQueryString information if I use the earlier version.
Do I change something in the configuration ? Any pointers will be
appreciated.
Thanks,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-4-7-not-showing-parsedQuery-parsedquery-toString-information-tp4132964.html
Sent
/ parsedQueryString information if I use the earlier version.
Do I change something in the configuration ? Any pointers will be
appreciated.
Thanks,
Shamik
for you to
look.
Regards,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/CollapsingQParserPlugin-returning-different-result-set-tp4123716p4125290.html
Sent from the Solr - User mailing list archive at Nabble.com.
Hi Joel, Thanks for taking a look into this. Here's the information you had
requested.*ADSKDedup:*I've attached separate files for debug information for
each query.Let me know if you need any information.Regards,Shamik
CollapsingQParserPlugin_Query_Debug.txt
http://lucene.472066.n3.nabble.com
.
I'll appreciate if someone can provide their feedback.
Regards,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/SolrCloud-inconsistent-result-for-the-same-query-tp4125005.html
Sent from the Solr - User mailing list archive at Nabble.com.
different ?
Any pointers will be appreciated.
-Thanks,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/CollapsingQParserPlugin-returning-different-result-set-tp4123716.html
Sent from the Solr - User mailing list archive at Nabble.com.
I'm missing here, any pointers will be appreciated.
- Thanks,
Shamik
Jack, thanks for the pointer. I should have checked this closely. I'm using
edismax and here's my qf entry :
str name=qf
id^10.0 cat^1.4 text^0.5 features^1.0 name^1.2 sku^1.5 manu^1.1
title^10.0 description^5.0 keywords^5.0 author^2.0 resourcename^1.0
/str
As you can see, I was
As Shawn had pointed, if you are using CloudSolrServer client, then you are
immune to the scenario where a shard and its replica(s) go down. The
communication should be ideally with the zookeepers and not the solr servers
directly, One thing you need to make sure is to add the shard.tolerant
. I'm using a SolrCloud environment with 2
shards having a replica each.
Any pointers will be appreciated.
Thanks,
Shamik
Thanks a lot Shawn. Changing the appends filtering based on your suggestion
worked. The part which confused me bigtime is the syntax I've been using so
far without an issue (barring the q.op part).
lst name=appends
str name=fqSource:TestHelp | Source:downloads |
-AccessMode:internal |
Thanks, I'll take a look at the debug data.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Weird-issue-with-q-op-AND-tp4117013p4117047.html
Sent from the Solr - User mailing list archive at Nabble.com.
remove stale content.
Let me know if I'm missing something here.
- Thanks,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/Indexing-question-on-individual-field-update-tp4116605p4116757.html
Sent from the Solr - User mailing list archive at Nabble.com.
Ok, I was wrong here. I can always set the indextimestamp field with current
time (NOW) for every atomic update. On a similar note, is there any
performance constraint with updates compared to add ?
--
View this message in context:
Thanks Eric and Shawn, appreciate your help.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Indexing-question-on-individual-field-update-tp4116605p4116831.html
Sent from the Solr - User mailing list archive at Nabble.com.
, which will keep the value if its already present in the index
for a given id ?
Any pointers will be appreciated.
Thanks,
Shamik
Thanks Joel, really appreciate your help. I'll keep an eye on the 4.6.1
release.
--
View this message in context:
http://lucene.472066.n3.nabble.com/SolrCloud-Result-Grouping-vs-CollapsingQParserPlugin-tp4111331p4111486.html
Sent from the Solr - User mailing list archive at Nabble.com.
and confined to a
dedicated shard. Just wondering if this applies to CollapsingQParserPlugin
as well ? Will result name=response *numFound=6* start=0 be
incorrect if the collapsed field is distributed ?
I'll really appreciate if someone can provide pointers on this.
Thanks,
Shamik
Joel,
Thanks for the pointer. I went through your blog on Document routing, very
informative. I do need some clarifications on the implementation. I'll try
to run it based on my use case.
I'm indexing documents from multiple source system out of which a bunch
consist of duplicate content. I'm
Thanks Joel, I found the issue. It had to do with the schema definition for
adskdedup field. I had defined it as a text_general which was analyzing it
based on -. After I changed it to type string, it worked as expected.
Thanks for looking into this.
--
View this message in context:
Hi,
I'm looking for some clarification on CollapsingQParserPlugin feature.
Here's what I tried. I downloaded 4.6, updated solr.xml under exampledocs
folder and added the following entry. I've added a new field adskdedup
on which I'm planning to test field collapsing. As you can see, out of
Thanks for the update Shawn, will look forward to the release.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-grouping-performance-porblem-tp4098565p4101314.html
Sent from the Solr - User mailing list archive at Nabble.com.
Thanks Joel, appreciate your help. Is Solr 4.6 due this year ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-grouping-performance-porblem-tp4098565p4100358.html
Sent from the Solr - User mailing list archive at Nabble.com.
about CollapsingQParserPlugin as an alternate to grouping, but that
seemed to be available in 4.6. Just wondering if it can be an alternate in
my case and whether if its possible to apply as a patch in 4.4 version.
Any pointer will be appreciated.
- Thanks,
Shamik
Bumping up this thread as I'm facing similar issue . Any solution ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Grouping-performance-problem-tp3995245p4098566.html
Sent from the Solr - User mailing list archive at Nabble.com.
Thanks for the information. I think its good to have this issue fixed,
specially for cases where the spellcheck feature is on. I'll check out at
the source code and take a look, even a quick suppressing of the null
pointer exception might make a difference.
--
View this message in context:
-around to
address this issue ? Any pointers will be appreciated.
Thanks,
Shamik
Thanks Primoz, I was suspecting that too. But then, its hard to imagine that
query cache is only contributing to the big performance hit. The setting
applies to the old configuration, and it works pretty well even with the
query cache low hit rate.
--
View this message in context:
I tried commenting out NOW in bq, but didn't make any difference in the
performance. I do see minor entry in the queryfiltercache rate which is a
meager 0.02.
I'm really struggling to figure out the bottleneck, any known pain points I
should be checking ?
--
View this message in context:
, can this
contribute ?
Sorry about the long post, but I'm struggling to nail down the issue here,
especially when queries are running fine in a master-slave environment with
similar hardware and network.
Any pointers will be highly appreciated.
Regards,
Shamik
--
View this message in context
the issue here,
especially when queries are running fine in a master-slave environment with
similar hardware and network.
Any pointers will be highly appreciated.
Regards,
Shamik
James,
Thanks for your reply. The shards.qt did the trick. I read the
documentation earlier but was not clear on the implementation, now it
totally makes sense.
Appreciate your help.
Regards,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/RE-How-to-achieve
101 - 200 of 209 matches
Mail list logo