Re: timeAllowed flag in the response

2012-06-08 Thread Michael Kuhlmann
Am 08.06.2012 11:55, schrieb Laurent Vaills: Hi Michael, Thanks for the details that helped me to take a deeper look in the source code. I noticed that each time a TimeExceededException is caught the method setPartialResults(true) is called...which seems to be what I'm looking for. I have to

Re: what's better for in memory searching?

2012-06-11 Thread Michael Kuhlmann
Set the swapiness to 0 to avoid memory pages being swapped to disk too early. http://en.wikipedia.org/wiki/Swappiness -Kuli Am 11.06.2012 10:38, schrieb Li Li: I have roughly read the codes of RAMDirectory. it use a list of 1024 byte arrays and many overheads. But as far as I know, using

Re: what's better for in memory searching?

2012-06-11 Thread Michael Kuhlmann
You cannot guarantee this when you're running out of RAM. You'd have a problem then anyway. Why are you caring that much? Did you yet have performance issues? 1GB should load really fast, and both auto warming and OS cache should help a lot as well. With such an index, you usually don't need

Re: Starts with Query

2012-06-15 Thread Michael Kuhlmann
It's not necessary to do this. You can simply be happy about the fact that all digits are ordered strictly in unicode, so you can use a range query: (f)q={!frange l=0 u=\: incl=true incu=false}title This finds all documents where any token from the title field starts with a digit, so if you

Re: Urgent: Facetable but not Searchable Field

2012-08-01 Thread Michael Kuhlmann
On 01.08.2012 13:58, jayakeerthi s wrote: We have a requirement, where we need to implement 2 fields as Facetable, but the values of the fields should not be Searchable. Simply don't search for it, then it's not searchable. Or do I simply don't understand your question? As long as Dismax

Re: Urgent: Facetable but not Searchable Field

2012-08-01 Thread Michael Kuhlmann
On 01.08.2012 15:40, Jack Krupansky wrote: The indexed and stored field attributes are independent, so you can define a facet field as stored but not indexed (stored=true indexed=false), so that the field can be faceted but not indexed. ? A field must be indexed to be used for faceting.

Re: SOLR 3.4 GeoSpatial Query Returning distance

2012-08-02 Thread Michael Kuhlmann
On 02.08.2012 01:52, Anand Henry wrote: Hi, In SOLR 3.4, while doing a geo-spatial search, is there a way to retrieve the distance of each document from the specified location? Not that I know of. What we did was to read and parse the location field on client side and calculate the distance

Re: Connect to SOLR over socket file

2012-08-08 Thread Michael Kuhlmann
On 07.08.2012 21:43, Jason Axelson wrote: Hi, Is it possible to connect to SOLR over a socket file as is possible with mysql? I've looked around and I get the feeling that I may be mi-understanding part of SOLR's architecture. Any pointers are welcome. Thanks, Jason Hi Jason, not that I

Re: Does Solr support 'Value Search'?

2012-08-09 Thread Michael Kuhlmann
On 08.08.2012 20:56, Bing Hua wrote: Not quite understand but I'd explain the problem I had. The response would contain only fields and a list of field values that match the query. Essentially it's querying for field values rather than documents. The underlying use case would be, when typing in

Re: how to sort search results by count matches

2016-08-02 Thread Michael Kuhlmann
Hi syegorius, are you sure that there's no synonym "planet,world" defined? -Michael Am 02.08.2016 um 15:57 schrieb syegorius: > I have 4 records index by Solr: > > 1 hello planet dear friends > 2 hello world dear friends > 3 nothing > 4 just friends > > I'm searching with this query: > >

Re: Sorl 6 with jetty issues

2017-02-20 Thread Michael Kuhlmann
This may be related to SOLR-10130. Am 20.02.2017 um 14:06 schrieb ~$alpha`: > Issues with solr settings while migrating from solr 4.0 to solr6.0. > > Issue Faced: My CPU consumption goes to unacceptable levels. ie. load on > solr4.0 is between 6 to 10 while the load on solr 6 reaches 100 and

Re: Select TOP 10 items from Solr Query

2017-02-16 Thread Michael Kuhlmann
So basically you want faceting only on the returned result set? I doubt that this is possible without additional queries. The issue is that faceting and result collecting is done within one iteration, so when some document (actually the document's internal id) is fetched as a possible result

Re: Continual garbage collection loop

2017-02-15 Thread Michael Kuhlmann
y 2017 at 14:44 Michael Kuhlmann <k...@solr.info> wrote: >> >> >> Wow, running 36 cores with only half a gigabyte of heap memory is >> *really* optimistic! >> >> I'd raise the heap size to some gigabytes at least and see how it's >> workin

Re: Continual garbage collection loop

2017-02-14 Thread Michael Kuhlmann
Wow, running 36 cores with only half a gigabyte of heap memory is *really* optimistic! I'd raise the heap size to some gigabytes at least and see how it's working then. -Michael Am 14.02.2017 um 15:23 schrieb Leon STRINGER: > Further background on the environment: > > There are 36 cores, with a

Re: Select TOP 10 items from Solr Query

2017-02-17 Thread Michael Kuhlmann
set > of the itemNo from the 1st query. > > There's definitely more than the 10, but we just need the top 10 in this > case. As the top 10 itemNo may change, so we have to get the returned > result set of the itemNo each time we want to do the JSON Facet. > > Regards, > Edwin

Re: Select TOP 10 items from Solr Query

2017-02-17 Thread Michael Kuhlmann
It's not possible to do such thing in one request with faceting only. The problem is that you need a fixed filter on every item when the facet algorithm is iterating over it; you can't look into future elements to find out which ones the top 10 will be. So either you stick with two queries (which

Re: Multi word synonyms

2016-11-15 Thread Michael Kuhlmann
It's not working out of the box, sorry. We're using this plugin: https://github.com/healthonnet/hon-lucene-synonyms#getting-started It's working nicely, but can lead to OOME when you add many synonyms with multiple terms. And I'm not sure whether it#s still working with Solr 6.0. -Michael Am

Re: Again : Query formulation help

2016-11-24 Thread Michael Kuhlmann
Hi Prasanna, there's no such filter out-of-the-box. It's similar to the mm parameter in (e)dismax parser, but this only works for full text searches on the same fields. So you have to build the query on your own using all possible permutations: fq=(code1: AND code2:) OR (code1: AND

Re: Multi word synonyms

2016-11-15 Thread Michael Kuhlmann
Hi Midas, > > I suggest this interesting reading: > > https://lucidworks.com/blog/2014/07/12/solution-for-multi-term-synonyms-in-lucenesolr-using-the-auto-phrasing-tokenfilter/ > > > > On Tue, Nov 15, 2016 at 11:00 AM, Michael Kuhlmann <k...@solr.info> wrote: >

Re: Multi word synonyms

2016-11-15 Thread Michael Kuhlmann
s > been closed and is available in 6.2. > > On Tue, Nov 15, 2016 at 12:32 PM, Michael Kuhlmann <k...@solr.info> wrote: > >> This is a nice reading though, but that solution depends on the >> precondition that you'll already know your synonyms at index time. >&

Re: File system choices?

2016-12-15 Thread Michael Kuhlmann
Yes, and we're doing such things at my company. However we most often do things you shouldn't do; this is one of these. Solr needs to load data quite fast, otherwise you'll be having a performance killer. It's often recommended to use an SSD instead of a normal hard disk; a network share would be

Re: FacetField-Result on String-Field contains value with count 0?

2017-01-13 Thread Michael Kuhlmann
Then I don't understand your problem. Solr already does exactly what you want. Maybe the problem is different: I assume that there never was a value of "1" in the index, leading to your confusion. Solr returns all fields as facet result where there was some value at some time as long as the the

Re: Solr Suggester

2016-12-22 Thread Michael Kuhlmann
For the suggester, the field must be indexed. It's not necessary to have it stored. Best, Michael Am 22.12.2016 um 11:24 schrieb Furkan KAMACI: > Hi Emir, > > As far as I know, it should be enough to be stored=true for a suggestion > field? Should it be both indexed and stored? > > Kind Regards,

Re: fq performance

2017-03-16 Thread Michael Kuhlmann
First of all, from what I can see, this won't do what you're expecting. Multiple fq conditions are always combined using AND, so if a user is member of 100 groups, but the document is accessible to only 99 of them, then the user won't find it. Or in other words, if you add a user to some

Re: fq performance

2017-03-17 Thread Michael Kuhlmann
Hi Ganesh, you might want to use something like this: fq=access_control:(g1 g2 g5 g99 ...) Then it's only one fq filter per request. Internally it's like an OR condition, but in a more condensed form. I already have used this with up to 500 values without larger performance degradation (but

Re: Mixing AND OR conditions with query parameters

2017-04-24 Thread Michael Kuhlmann
Make sure to have a whitespace are the OR operator. The parenthesises should be around the OR query, not including the "fq:" -- this should be outside the parenthesises (which are not necessary at all). What exactly are you expecting? -Michael Am 24.04.2017 um 12:59 schrieb VJ: > Hi All, > > I

Re: Error after moving index

2017-06-22 Thread Michael Kuhlmann
Hi Moritz, did you stop your local Solr sever before? Copying data from a running instance may cause headaches. If yes, what happens if you copy everything again? It seems that your copy operations wasn't successful. Best, Michael Am 22.06.2017 um 14:37 schrieb Moritz Munte: > Hello, > > > >

Re: Solr NLS custom query parser

2017-06-15 Thread Michael Kuhlmann
Hi Arun, your question is too generic. What do you mean with nlp search? What do you expect to happen? The short answer is: No, there is no such parser because the individual requirements will vary a lot. -Michael Am 14.06.2017 um 16:32 schrieb aruninfo100: > Hi, > > I am trying to configure

Re: solr Facet.contains

2017-09-15 Thread Michael Kuhlmann
What is the field type? Which Analyzers are configured? How do you split at "~"? (You have to do it by yourself, or configure some tokenizer for that.) What do you get when you don't filter your facets? What do you mean with "it is not working"? What is your result now? -Michael  Am 15.09.2017

Re: Solr nodes crashing (OOM) after 6.6 upgrade

2017-09-22 Thread Michael Kuhlmann
Hi Shamik, funny enough, we had a similar issue with our old legacy application that still used plain Lucene code in a JBoss container. Same, there were no specific queries or updates causing this, the performance just broke completely without unusual usage. GC was raising up to 99% or so.

Re: Newbie question about why represent timestamps as "float" values

2017-10-10 Thread Michael Kuhlmann
While you're generally right, in this case it might make sense to stick to a primitive type. I see "unixtime" as a technical information, probably from System.currentTimeMillis(). As long as it's not used as a "real world" date but only for sorting based on latest updates, or chosing which

Re: Solr Issue

2017-09-07 Thread Michael Kuhlmann
Hi Patrick, can you attach the query you're sending to Solr and one example result? Or more specific, what are your hl.* parameters? -Michael Am 07.09.2017 um 09:36 schrieb Patrick Fallert: > > Hey Guys,  > i´ve got a problem with my Solr Highlighter.. > When I search for a word, i get some

Re: ways to check if document is in a huge search result set

2017-09-11 Thread Michael Kuhlmann
Maybe I don't understand your problem, but why don't you just filter by "supplier information"? -Michael Am 11.09.2017 um 04:12 schrieb Derek Poh: > Hi > > I have a collection of productdocument. > Each productdocument has supplier information in it. > > I need to check if a supplier's products

Re: Solr6.6 Issue/Bug

2017-09-06 Thread Michael Kuhlmann
Why would you need to start Solr as root? You should definitely not do this, there's no reason for that. And even if you *really* want this: What's so bad about the -force option? -Michael Am 06.09.2017 um 07:26 schrieb Kasim Jinwala: > Dear team, > I am using solr 5.0 last 1 year,

Re: ways to check if document is in a huge search result set

2017-09-12 Thread Michael Kuhlmann
So you're looking for a solution to validate the result output. You have two ways: 1. Assuming you're sorting by the default "score" sort option: Find the result you're looking for by setting the fq filter clause accordingly, and add "score" the the fl field list. Then do the normal unfiltered

Re: ways to check if document is in a huge search result set

2017-09-13 Thread Michael Kuhlmann
Am 13.09.2017 um 04:04 schrieb Derek Poh: > Hi Michael > > "Then continue using binary search depending on the returned score > values." > > May I know what do you mean by using binary search? An example algorithm is in Java method java.util.Arrays::binarySearch. Or more detailed:

Re: Modifing create_core's instanceDir attribute

2017-09-28 Thread Michael Kuhlmann
I'd rather say you didn't quote the URL when sending it using curl. Bash accepts the ampersand as a request to execute curl including the URL up to CREATE in background - that's why the error is included within the next output, followed by "Exit" - and then tries to execute the following part of

Re: Where the uploaded configset from SOLR into zookeeper ensemble resides?

2017-09-28 Thread Michael Kuhlmann
Do you find your configs in the Solr admin panel, in the Cloud --> Tree folder? -Michael Am 28.09.2017 um 04:50 schrieb Gunalan V: > Hello, > > Could you please let me know where can I find the uploaded configset from > SOLR into zookeeper ensemble ? > > In docs it says they will "/configs/"

Re: Moving to Point, trouble with IntPoint.newRangeQuery()

2017-09-26 Thread Michael Kuhlmann
Hi Markus, I don't know why there aren't any results. But just out of curiosity, why don't you use the better choice IntPoint.newExectQuery(String,int)? What happens if you use that? -Michael Am 26.09.2017 um 13:22 schrieb Markus Jelsma: > Hello, > > I have a QParser impl. that transforms

Re: Moving to Point, trouble with IntPoint.newRangeQuery()

2017-09-26 Thread Michael Kuhlmann
Arrgh, forget my question. I just see that newExactQuery() simply triggers newRangeQuery() like you already do. -Michael Am 26.09.2017 um 13:29 schrieb Michael Kuhlmann: > Hi Markus, > > I don't know why there aren't any results. But just out of curiosity, > why don't you use the b

Re: How to sort on dates?

2017-12-18 Thread Michael Kuhlmann
Am 16.12.2017 um 19:39 schrieb Georgios Petasis: > Even if the DateRangeField field can store a range of dates, doesn't > Solr understand that I have used single timestamps? No. It could theoretically, but sorting just isn't implemented in DateRangeField. > I have even stored the dates. > My

Re: Wildcard searches with special character gives zero result

2017-12-15 Thread Michael Kuhlmann
Solr does not analyze queries with wildcards in it. So, with ch*p-seq, it will search for terms that start with ch and end with p-seq. Since your indexer has analyzed all tokens before, only chip and seq are in the index. See

Re: How to sort on dates?

2017-12-15 Thread Michael Kuhlmann
Hi Georgios, DateRangeField is a kind of SpatialField which is not sortable at all. For sorting, use a DatePointField instead. It's not deprecated; the deprecated class is TrieDateField. Best, Michael Am 15.12.2017 um 10:53 schrieb Georgios Petasis: > Hi all, > > I have a field of type

Re: SolrException undefined field *

2018-01-09 Thread Michael Kuhlmann
help you better when you pass the full query string (if you're able to fetch it). -Michael Am 09.01.2018 um 16:38 schrieb Michael Kuhlmann: > First, you might want to index, but what Solr is executing here is a > search request. > > Second, you're querying for a dynamic field

Re: SolrException undefined field *

2018-01-09 Thread Michael Kuhlmann
First, you might want to index, but what Solr is executing here is a search request. Second, you're querying for a dynamic field "*" which is not defined in your schema. This is quite obvious, the exception says right this. So whatever is sending the query (some client, it seems) is doing the

Re: Edismax leading wildcard search

2017-12-22 Thread Michael Kuhlmann
Am 22.12.2017 um 11:57 schrieb Selvam Raman: > 1) how can i disable leading wildcard search Do it on the client side. Just don't allow leading asterisks or question marks in your query term. > 2) why leading wildcard search takes so much of time to give the response. > Because Lucene can't

Re: How to split index more than 2GB in size

2018-06-21 Thread Michael Kuhlmann
Hi Sushant, while this is true in general, it won't hold here. If you split your index, searching on each splitted shard might be a bit faster, but you'll increase search time much more because Solr needs to send your search queries to all shards and then combine the results. So instead of having

Re: Query with exact number of tokens

2018-09-21 Thread Michael Kuhlmann
Hi Sergio, alas that's not possible that way. If you search for CENTURY BANCORP, INC., then Solr will be totally happy to find all these terms in "NEW CENTURY BANCORP, INC." and return it with a high score. But you can prepare your data at index time. Make it a multivalued field of type string

<    1   2