Re: Solr 4.10.4 export handler NPE
Thank you Eric for the reply. Not possible to change from version 4.10.4, we have built lot of functionalities wrapping 4.10 version. I have decided to use /select handler to fetch data incrementally and write to a file. I think that will work Regards, Lasitha On 2 Aug 2017 23:37, "Erick Erickson" <erickerick...@gmail.com> wrote: > That the JIRA says 5.5 just means that's the version the submitter was > using when the NPE was encountered. No workarounds that I know of, but > that code has changed drastically since the 4.10 days so I really have > no clue. Any chance of trying it on a 6.6 release? > > Best, > Erick > > On Tue, Aug 1, 2017 at 8:43 PM, Lasitha Wattaladeniya <watt...@gmail.com> > wrote: > > Hi devs, > > > > I was exploring the /export handler in solr and got an exception. When I > > research online I found this open jira case : SOLR-8860 > > > > https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-8806 > > > > is this a valid jira case? Any workarounds? > > > > Jira says affect version is 5.5 but I'm getting this in 4.10.4 also > > > > > > Regards, > > Lasitha >
Solr 4.10.4 export handler NPE
Hi devs, I was exploring the /export handler in solr and got an exception. When I research online I found this open jira case : SOLR-8860 https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-8806 is this a valid jira case? Any workarounds? Jira says affect version is 5.5 but I'm getting this in 4.10.4 also Regards, Lasitha
Re: Highlighting words with special characters
Hi Shawn, Yes I can confirm, it works with out any errors with multiple tokenizers. Following is my analysis chain StandardTokenizerFactory (only in index) StopFilterFactory LowerCaseFilterFactory ASCIIFoldingFilterFactory EnglishPossessiveFilterFactory StemmerOverrideFilterFactory (only in query) NgramTokenizerFactory (only in index) I'll have a look more into what you said, Single tokenizer in analysis chain. Regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Thu, Jul 20, 2017 at 7:12 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 7/19/2017 8:31 PM, Lasitha Wattaladeniya wrote: > > But I have NgramTokenizerFactory at the end of indexing analyzer chain. > > Therefore I should still tokenize the email address. But how this affects > > the highlighting?, that's what I'm confused to understand > > You can only have one tokenizer in an analysis chain. I have no idea > what happens if you have more than one. I personally would expect that > to result in an initialization error, but maybe what it does is ignore > the additional tokenizers. Your experience seems to indicate that it > does NOT result in an error. Can you confirm? > > The analysis is done in this order: > > CharFilters > Tokenizer > Filters > > Thanks, > Shawn > >
Re: Highlighting words with special characters
Hi ahmet, But I have NgramTokenizerFactory at the end of indexing analyzer chain. Therefore I should still tokenize the email address. But how this affects the highlighting?, that's what I'm confused to understand Solr version : 4.10.4 Regards, Lasitha On 20 Jul 2017 08:28, "Ahmet Arslan" <iori...@yahoo.com.invalid> wrote: Hi, Maybe name of the UAX29URLEMailTokenizer is deceiving you?It does *not* tokenize URLs and Emails. Actually it recognises them and emits them as a single token. Ahmet On Wednesday, July 19, 2017, 12:00:05 PM GMT+3, Lasitha Wattaladeniya < watt...@gmail.com> wrote: Update, I changed the UAX29URLEmailTokenizerFactory to StandardTokenizerFactory and now it shows highlighted text fragments in the indexed email text. But I don't understand this behavior. Can someone shed some light please On 18 Jul 2017 14:18, "Lasitha Wattaladeniya" <watt...@gmail.com> wrote: > Further more, ngram field has following tokenizer/filter chain in index > and query > > UAX29URLEmailTokenizerFactory (only in index) > stopFilterFactory > LowerCaseFilterFactory > ASCIIFoldingFilterFactory > EnglishPossessiveFilterFactory > StemmerOverrideFilterFactory (only in query) > NgramTokenizerFactory (only in index) > > Regards, > Lasitha > > On 18 Jul 2017 14:11, "Lasitha Wattaladeniya" <watt...@gmail.com> wrote: > >> Hi devs, >> >> I have setup solr highlighting with default setup (only changed the >> fragsize to 0 to match any field length). It worked fine but recently I >> discovered it doesn't highlight for words with special characters in the >> middle. >> >> For an example, let's say I have indexed email address test.f...@ran.com >> to a ngram field. And when I search for the partial text fsdg, I get the >> results but it's not highlighted. It works in all other scenarios as >> expected. >> >> The ngram field has termVectors, termPositions, termOffsets set to true. >> >> Can somebody please suggest me, what may be wrong here? >> >> (sorry for the unstructured text. Typed using a mobile phone ) >> >> Regards >> Lasitha >> >
Re: Highlighting words with special characters
Update, I changed the UAX29URLEmailTokenizerFactory to StandardTokenizerFactory and now it shows highlighted text fragments in the indexed email text. But I don't understand this behavior. Can someone shed some light please On 18 Jul 2017 14:18, "Lasitha Wattaladeniya" <watt...@gmail.com> wrote: > Further more, ngram field has following tokenizer/filter chain in index > and query > > UAX29URLEmailTokenizerFactory (only in index) > stopFilterFactory > LowerCaseFilterFactory > ASCIIFoldingFilterFactory > EnglishPossessiveFilterFactory > StemmerOverrideFilterFactory (only in query) > NgramTokenizerFactory (only in index) > > Regards, > Lasitha > > On 18 Jul 2017 14:11, "Lasitha Wattaladeniya" <watt...@gmail.com> wrote: > >> Hi devs, >> >> I have setup solr highlighting with default setup (only changed the >> fragsize to 0 to match any field length). It worked fine but recently I >> discovered it doesn't highlight for words with special characters in the >> middle. >> >> For an example, let's say I have indexed email address test.f...@ran.com >> to a ngram field. And when I search for the partial text fsdg, I get the >> results but it's not highlighted. It works in all other scenarios as >> expected. >> >> The ngram field has termVectors, termPositions, termOffsets set to true. >> >> Can somebody please suggest me, what may be wrong here? >> >> (sorry for the unstructured text. Typed using a mobile phone ) >> >> Regards >> Lasitha >> >
Re: Highlighting words with special characters
Further more, ngram field has following tokenizer/filter chain in index and query UAX29URLEmailTokenizerFactory (only in index) stopFilterFactory LowerCaseFilterFactory ASCIIFoldingFilterFactory EnglishPossessiveFilterFactory StemmerOverrideFilterFactory (only in query) NgramTokenizerFactory (only in index) Regards, Lasitha On 18 Jul 2017 14:11, "Lasitha Wattaladeniya" <watt...@gmail.com> wrote: > Hi devs, > > I have setup solr highlighting with default setup (only changed the > fragsize to 0 to match any field length). It worked fine but recently I > discovered it doesn't highlight for words with special characters in the > middle. > > For an example, let's say I have indexed email address test.f...@ran.com > to a ngram field. And when I search for the partial text fsdg, I get the > results but it's not highlighted. It works in all other scenarios as > expected. > > The ngram field has termVectors, termPositions, termOffsets set to true. > > Can somebody please suggest me, what may be wrong here? > > (sorry for the unstructured text. Typed using a mobile phone ) > > Regards > Lasitha >
Highlighting words with special characters
Hi devs, I have setup solr highlighting with default setup (only changed the fragsize to 0 to match any field length). It worked fine but recently I discovered it doesn't highlight for words with special characters in the middle. For an example, let's say I have indexed email address test.f...@ran.com to a ngram field. And when I search for the partial text fsdg, I get the results but it's not highlighted. It works in all other scenarios as expected. The ngram field has termVectors, termPositions, termOffsets set to true. Can somebody please suggest me, what may be wrong here? (sorry for the unstructured text. Typed using a mobile phone ) Regards Lasitha
Java bigdecimal equivalent field type in solr
Hi devs, Can you please help me to identify what is the java bigdecimal equivalent field type in solr. Currently I was indexing the float value of the bigdecimal to a *_f field. But it looses precision when I call the Bigdecimal.floatValue() method Regards, Lasitha
Re: HttpSolrServer default connection timeout and socket timeout
Hi devs, I tried out HttpSolrServer.setConnectionTimeout() method. But im getting java.lang.UnsupportedException error in my logs. After a littlebit research I found this issue, https://issues.apache.org/jira/browse/SOLR-6542 Any workarounds for this issue? Regards, Lasitha On 16 Mar 2017 09:48, "Lasitha Wattaladeniya" <watt...@gmail.com> wrote: > Hi devs, > > What are the default HttpSolrServer connection timeout and socket timeout > used? > > And what are the recommended values we should use? > > My system is having some problems when the load is high, just trying to > figure out what are the correct values should set > > Regards, > Lasitha >
Solrj HttpSolrServer retryHandler
Hi folks, Is there a API to implement a retryHandler in HttpSolrServer ? I'm using solrj 4.10.4 Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com
HttpSolrServer default connection timeout and socket timeout
Hi devs, What are the default HttpSolrServer connection timeout and socket timeout used? And what are the recommended values we should use? My system is having some problems when the load is high, just trying to figure out what are the correct values should set Regards, Lasitha
Re: ConcurrentUpdateSolrServer update response doesn't reflect correct response status
Hi Shawn, Thanks for the detailed explanation, really informative. However after further analyzing I found that I can use HttpSolrServer instead of ConcurrentUpdateSolrServer, I'm handling the concurrency by my self. But the issue with ConcurrentUpdateSolrServer is still there, seem like an interesting topic to look into. Regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Mon, Feb 13, 2017 at 10:01 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 2/12/2017 10:51 PM, Lasitha Wattaladeniya wrote: > > Thanks for the reply. But even if I add a single document, the response > > doesn't reflect the correct response status. From the given solr console > > ui, it shows status 400 for the bad requests (when I try to index > document > > with no fields ). But when I do the same using solrj, > response > > is 0 eventhough the console prints it as a bad request with status 400. > > The concurrent client swallows all indexing exceptions. This is how it > is designed. In fact, it's potentially even worse than you might expect > -- the client not only always returns a success for updates sent, it > returns immediately, often before any updates are sent to the server. > The updates are sent in the background. If another request type is > sent, like a commit, then the client will wait for all queued updates to > be sent before it sends that request and returns. If an error is > encountered during an explicit commit, I believe that the client *is* > notified about that, but I have not tested this. > > If you want multi-threaded indexing *and* error detection, you're going > to need to handle the multiple threads in your own program, using the > Http or Cloud client object instead. The Concurrent object is good for > initial bulk indexing, but not for the kind of indexing where you want > to be informed about errors. > > I'm told that if you override the "handleError" method, you might be > able to make it work how you want, but I was unable to figure out how to > modify that method to achieve that result. > > I filed an issue and even came up with an idea for a fix, but nobody was > excited about it. Because of the design of the client, it wasn't > possible to guarantee that no further indexing had occurred after the > first error encountered. > > https://issues.apache.org/jira/browse/SOLR-3284 > > Thanks, > Shawn > >
Re: ConcurrentUpdateSolrServer update response doesn't reflect correct response status
Hi Eric, Thanks for the reply. But even if I add a single document, the response doesn't reflect the correct response status. From the given solr console ui, it shows status 400 for the bad requests (when I try to index document with no fields ). But when I do the same using solrj, response is 0 eventhough the console prints it as a bad request with status 400. P.S: I'm using solrj 4.10.4 Regards, Lasitha On 13 Feb 2017 13:33, "Erick Erickson" <erickerick...@gmail.com> wrote: Closest that I know of is: https://issues.apache.org/jira/browse/SOLR-445, available since Solr 6.1 Best, Erick On Sun, Feb 12, 2017 at 8:05 PM, Lasitha Wattaladeniya <watt...@gmail.com> wrote: > Hi devs, > > I'm using the ConcurrentUpdateSolrServer to add some documents to solr. > While I'm doing so I see some 'bad request" errors printed in the console > due to incorrect solrinputdocuments. > > The issue is I can't capture this issue by reading updateresonse status. > I saw some one had stated that this is due to ConcurrentUpdateSolrServer > uses a queue to buffer the documents and then later call update method. > > Isn't there any way to capture the correct response? > > Regards, > Lasitha
ConcurrentUpdateSolrServer update response doesn't reflect correct response status
Hi devs, I'm using the ConcurrentUpdateSolrServer to add some documents to solr. While I'm doing so I see some 'bad request" errors printed in the console due to incorrect solrinputdocuments. The issue is I can't capture this issue by reading updateresonse status. I saw some one had stated that this is due to ConcurrentUpdateSolrServer uses a queue to buffer the documents and then later call update method. Isn't there any way to capture the correct response? Regards, Lasitha
Re: Soft commit and reading data just after the commit
Hi Hendrik, Thanks for your input. Previously I was using the hard commit (SolrClient.commit()) but then I got some error when there are concurrent real time index requests from my app. The error was "Exceeded limit of maxWarmingSearchers=2, try again later", then i changed the code to use only solrserver.add(docs) method and configured autoSoftCommit timeout and autoCommit timeout in solrConfig. I think, i'll get the same error when I use the method you described (SolrClient.commit(String collection, boolean waitFlush, boolean waitSearcher, boolean softCommit)), Anyway i'll have a look at that method. Best regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 <+65%209389%206893> Blog : techreadme.blogspot.com On Tue, Dec 20, 2016 at 3:31 AM, Hendrik Haddorp <hendrik.hadd...@gmx.net> wrote: > Hi, > > the SolrJ API has this method: SolrClient.commit(String collection, > boolean waitFlush, boolean waitSearcher, boolean softCommit). > My assumption so far was that when you set waitSearcher to true that the > method call only returns once a search would find the new data, which > sounds what you want. I used this already and it seemed to work just fine. > > regards, > Hendrik > > > On 19.12.2016 04:09, Lasitha Wattaladeniya wrote: > >> Hi all, >> >> Thanks for your replies, >> >> @dorian : the requirement is, we are showing a list of entries on a page. >> For each user there's a read / unread flag. The data for listing is >> fetched from solr. And you can see the entry was previously read or not. >> So >> when a user views an entry by clicking. We are updating the database flag >> to READ and use real time indexing to update solr entry. So when the user >> close the full view of the entry and go back to entry listing page, the >> data fetched from solr should be updated to READ. That's the use case we >> are trying to fix. >> >> @eric : thanks for the lengthy reply. So let's say I increase the >> autosoftcommit time out to may be 100 ms. In that case do I have to wait >> much that time from client side before calling search ?. What's the >> correct way of achieving this? >> >> Regards, >> Lasitha >> >> On 18 Dec 2016 23:52, "Erick Erickson" <erickerick...@gmail.com> wrote: >> >> 1 ms autocommit is far too frequent. And it's not >>> helping you anyway. >>> >>> There is some lag between when a commit happens >>> and when the docs are really available. The sequence is: >>> 1> commit (soft or hard-with-opensearcher=true doesn't matter). >>> 2> a new searcher is opened and autowarming starts >>> 3> until the new searcher is opened, queries continue to be served by >>> the old searcher >>> 4> the new searcher is fully opened >>> 5> _new_ requests are served by the new searcher. >>> 6> the last request is finished by the old searcher and it's closed. >>> >>> So what's probably happening is that you send docs and then send a >>> query and Solr is still in step <3>. You can look at your admin UI >>> pluginst/stats page or your log to see how long it takes for a >>> searcher to open and adjust your expectations accordingly. >>> >>> If you want to fetch only the document (not try to get it by a >>> search), Real Time Get is designed to insure that you always get the >>> most recent copy whether it's searchable or not. >>> >>> All that said, Solr wasn't designed for autocommits that are that >>> frequent. That's why the documentation talks about _Near_ Real Time. >>> You may need to adjust your expectations. >>> >>> Best, >>> Erick >>> >>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com> >>> wrote: >>> >>>> There's a very high probability that you're using the wrong tool for the >>>> job if you need 1ms softCommit time. Especially when you always need it >>>> >>> (ex >>> >>>> there are apps where you need commit-after-insert very rarely). >>>> >>>> So explain what you're using it for ? >>>> >>>> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya < >>>> >>> watt...@gmail.com> >>> >>>> wrote: >>>> >>>> Hi Furkan, >>>>> >>>>> Thanks for the links. I had read the first one but not the second one. >>>>> I >>>>> did read it after you sent. So in my current solrconfig.xml settings >>&g
Re: Soft commit and reading data just after the commit
Hi Shawn, Thanks for your well detailed explanation. Now I understand, I won't be able to achieve the 100ms softcommit timeout with my hardware setup. However let's say someone has a requirement as below (quoted from my previous mail) *Requirement *is, we are showing a list of entries on a page. For each user there's a read / unread flag. The data for listing is fetched from solr. And you can see the entry was previously read or not. So when a user views an entry by clicking. We are updating the database flag to READ and use real time indexing to update solr index. So when the user close the full view of the entry and go back to entry listing page, the data fetched from solr should be updated to READ. Can't we achieve a requirement as described above using solr ? (without manipulating the previously fetched results list from solr, because at some point we'll have to go back to search results from solr and at that time it should be updated). Regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Mon, Dec 19, 2016 at 6:37 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 12/18/2016 7:09 PM, Lasitha Wattaladeniya wrote: > > @eric : thanks for the lengthy reply. So let's say I increase the > > autosoftcommit time out to may be 100 ms. In that case do I have to > > wait much that time from client side before calling search ?. What's > > the correct way of achieving this? > > Some of the following is covered by the links you've already received. > Some of it may be new information. > > Before you can see a change you've just made, you will need to wait for > the commit to be fired (in this case, the autoSoftCommit time) plus > however long it actually takes to complete the commit and open a new > searcher. Opening the searcher is the expensive part. > > What I typically recommend that people do is have the autoSoftCommit > time as long as they can stand, with 60-300 seconds as a "typical" > value. That's a setting of 6 to 30. What you are trying to > achieve is much faster, and much more difficult. > > 100 milliseconds will typically be far too small a value unless your > index is extremely small or your hardware is incredibly fast and has a > lot of memory. With a value of 100, you'll want each of those soft > commits (which do open a new searcher) to take FAR less than 100 > milliseconds to complete. This kind of speed can be difficult to > achieve, especially if the index is large. > > To have any hope of fast commit times, you will need to set > autowarmCount on all Solr caches to zero. If you are indexing > frequently enough, you might even want to completely disable Solr's > internal caches, because they may be providing no benefit. > > You will want to have enough extra memory that your operating system can > cache the vast majority (or even maybe all) of your index. > > https://wiki.apache.org/solr/SolrPerformanceProblems > > Some other info that's helpful for understanding why plenty of *spare* > memory (not allocated by programs) is necessary for good performance: > > https://en.wikipedia.org/wiki/Page_cache > http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html > > The reason in a nutshell: Disks are EXTREMELY slow. Memory is very fast. > > Thanks, > Shawn > >
Re: Soft commit and reading data just after the commit
I didn't look much onto REALTIME GET handler. Thanks for mentioning
that. I'm checking it now
On 19 Dec 2016 10:09, "Lasitha Wattaladeniya" <watt...@gmail.com> wrote:
> Hi all,
>
> Thanks for your replies,
>
> @dorian : the requirement is, we are showing a list of entries on a page.
> For each user there's a read / unread flag. The data for listing is
> fetched from solr. And you can see the entry was previously read or not. So
> when a user views an entry by clicking. We are updating the database flag
> to READ and use real time indexing to update solr entry. So when the user
> close the full view of the entry and go back to entry listing page, the
> data fetched from solr should be updated to READ. That's the use case we
> are trying to fix.
>
> @eric : thanks for the lengthy reply. So let's say I increase the
> autosoftcommit time out to may be 100 ms. In that case do I have to wait
> much that time from client side before calling search ?. What's the
> correct way of achieving this?
>
> Regards,
> Lasitha
>
> On 18 Dec 2016 23:52, "Erick Erickson" <erickerick...@gmail.com> wrote:
>
>> 1 ms autocommit is far too frequent. And it's not
>> helping you anyway.
>>
>> There is some lag between when a commit happens
>> and when the docs are really available. The sequence is:
>> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
>> 2> a new searcher is opened and autowarming starts
>> 3> until the new searcher is opened, queries continue to be served by
>> the old searcher
>> 4> the new searcher is fully opened
>> 5> _new_ requests are served by the new searcher.
>> 6> the last request is finished by the old searcher and it's closed.
>>
>> So what's probably happening is that you send docs and then send a
>> query and Solr is still in step <3>. You can look at your admin UI
>> pluginst/stats page or your log to see how long it takes for a
>> searcher to open and adjust your expectations accordingly.
>>
>> If you want to fetch only the document (not try to get it by a
>> search), Real Time Get is designed to insure that you always get the
>> most recent copy whether it's searchable or not.
>>
>> All that said, Solr wasn't designed for autocommits that are that
>> frequent. That's why the documentation talks about _Near_ Real Time.
>> You may need to adjust your expectations.
>>
>> Best,
>> Erick
>>
>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com>
>> wrote:
>> > There's a very high probability that you're using the wrong tool for the
>> > job if you need 1ms softCommit time. Especially when you always need it
>> (ex
>> > there are apps where you need commit-after-insert very rarely).
>> >
>> > So explain what you're using it for ?
>> >
>> > On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
>> watt...@gmail.com>
>> > wrote:
>> >
>> >> Hi Furkan,
>> >>
>> >> Thanks for the links. I had read the first one but not the second one.
>> I
>> >> did read it after you sent. So in my current solrconfig.xml settings
>> below
>> >> are the configurations,
>> >>
>> >>
>> >>${solr.autoSoftCommit.maxTime:1}
>> >>
>> >>
>> >>
>> >>
>> >>15000
>> >>false
>> >>
>> >>
>> >> The problem i'm facing is, just after adding the documents to solr
>> using
>> >> solrj, when I retrieve data from solr I am not getting the updated
>> results.
>> >> This happens time to time. Most of the time I get the correct data but
>> in
>> >> some occasions I get wrong results. so as you suggest, what the best
>> >> practice to use here ? , should I wait 1 mili second before calling for
>> >> updated results ?
>> >>
>> >> Regards,
>> >> Lasitha
>> >>
>> >> Lasitha Wattaladeniya
>> >> Software Engineer
>> >>
>> >> Mobile : +6593896893
>> >> Blog : techreadme.blogspot.com
>> >>
>> >> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com
>> >
>> >> wrote:
>> >>
>> >> > Hi Lasitha,
>> >> >
>> >> > First of all, did you check these:
>> >> >
>> >> > https://cwiki.apache.org/confluence/display/solr/Near+
>> >> Real+Time+
Re: Soft commit and reading data just after the commit
Hi all,
Thanks for your replies,
@dorian : the requirement is, we are showing a list of entries on a page.
For each user there's a read / unread flag. The data for listing is
fetched from solr. And you can see the entry was previously read or not. So
when a user views an entry by clicking. We are updating the database flag
to READ and use real time indexing to update solr entry. So when the user
close the full view of the entry and go back to entry listing page, the
data fetched from solr should be updated to READ. That's the use case we
are trying to fix.
@eric : thanks for the lengthy reply. So let's say I increase the
autosoftcommit time out to may be 100 ms. In that case do I have to wait
much that time from client side before calling search ?. What's the
correct way of achieving this?
Regards,
Lasitha
On 18 Dec 2016 23:52, "Erick Erickson" <erickerick...@gmail.com> wrote:
> 1 ms autocommit is far too frequent. And it's not
> helping you anyway.
>
> There is some lag between when a commit happens
> and when the docs are really available. The sequence is:
> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
> 2> a new searcher is opened and autowarming starts
> 3> until the new searcher is opened, queries continue to be served by
> the old searcher
> 4> the new searcher is fully opened
> 5> _new_ requests are served by the new searcher.
> 6> the last request is finished by the old searcher and it's closed.
>
> So what's probably happening is that you send docs and then send a
> query and Solr is still in step <3>. You can look at your admin UI
> pluginst/stats page or your log to see how long it takes for a
> searcher to open and adjust your expectations accordingly.
>
> If you want to fetch only the document (not try to get it by a
> search), Real Time Get is designed to insure that you always get the
> most recent copy whether it's searchable or not.
>
> All that said, Solr wasn't designed for autocommits that are that
> frequent. That's why the documentation talks about _Near_ Real Time.
> You may need to adjust your expectations.
>
> Best,
> Erick
>
> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com>
> wrote:
> > There's a very high probability that you're using the wrong tool for the
> > job if you need 1ms softCommit time. Especially when you always need it
> (ex
> > there are apps where you need commit-after-insert very rarely).
> >
> > So explain what you're using it for ?
> >
> > On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
> watt...@gmail.com>
> > wrote:
> >
> >> Hi Furkan,
> >>
> >> Thanks for the links. I had read the first one but not the second one. I
> >> did read it after you sent. So in my current solrconfig.xml settings
> below
> >> are the configurations,
> >>
> >>
> >>${solr.autoSoftCommit.maxTime:1}
> >>
> >>
> >>
> >>
> >>15000
> >>false
> >>
> >>
> >> The problem i'm facing is, just after adding the documents to solr using
> >> solrj, when I retrieve data from solr I am not getting the updated
> results.
> >> This happens time to time. Most of the time I get the correct data but
> in
> >> some occasions I get wrong results. so as you suggest, what the best
> >> practice to use here ? , should I wait 1 mili second before calling for
> >> updated results ?
> >>
> >> Regards,
> >> Lasitha
> >>
> >> Lasitha Wattaladeniya
> >> Software Engineer
> >>
> >> Mobile : +6593896893
> >> Blog : techreadme.blogspot.com
> >>
> >> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com>
> >> wrote:
> >>
> >> > Hi Lasitha,
> >> >
> >> > First of all, did you check these:
> >> >
> >> > https://cwiki.apache.org/confluence/display/solr/Near+
> >> Real+Time+Searching
> >> > https://lucidworks.com/blog/2013/08/23/understanding-
> >> > transaction-logs-softcommit-and-commit-in-sorlcloud/
> >> >
> >> > after that, if you cannot adjust your configuration you can give more
> >> > information and we can find a solution.
> >> >
> >> > Kind Regards,
> >> > Furkan KAMACI
> >> >
> >> > On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
> >> watt...@gmail.com>
> >> > wrote:
> >> >
> >> >> Hi furkan,
> >> >>
> >> >> Thank
Re: Stemming with SOLR
Thank you all for the replies. I am considering the suggestions On 17 Dec 2016 01:50, "Susheel Kumar" <susheel2...@gmail.com> wrote: > To handle irregular nouns ( > http://www.ef.com/english-resources/english-grammar/ > singular-and-plural-nouns/), > the simplest way is handle them using StemOverriderFactory. The list is > not so long. Or otherwise go for commercial solutions like basistech etc. > as Alex suggested oR you can customize Hunspell extensively to handle most > of them. > > Thanks, > Susheel > > On Thu, Dec 15, 2016 at 9:46 PM, Alexandre Rafalovitch <arafa...@gmail.com > > > wrote: > > > If you need the full fidelity solution taking care of multiple > > edge-cases, it could be worth looking at commercial solutions. > > > > > > http://www.basistech.com/ has one, including a free-level SAAS plan. > > > > Regards, > >Alex. > > > > http://www.solr-start.com/ - Resources for Solr users, new and > experienced > > > > > > On 15 December 2016 at 21:28, Lasitha Wattaladeniya <watt...@gmail.com> > > wrote: > > > Hi all, > > > > > > Thanks for the replies, > > > > > > @eric, ahmet : since those stemmers are logical stemmers it won't work > on > > > words such as caught, ran and so on. So in our case it won't work > > > > > > @susheel : Yes I thought about it but problems we have is, the > documents > > we > > > index are some what large text, so copy fielding these into duplicate > > > fields will affect on the index time ( we have jobs to index data > > > periodically) and query time. I wonder why there isn't a correct > solution > > > to this > > > > > > Regards, > > > Lasitha > > > > > > Lasitha Wattaladeniya > > > Software Engineer > > > > > > Mobile : +6593896893 > > > Blog : techreadme.blogspot.com > > > > > > On Fri, Dec 16, 2016 at 12:58 AM, Susheel Kumar <susheel2...@gmail.com > > > > > wrote: > > > > > >> We did extensive comparison in the past for Snowball, KStem and > Hunspell > > >> and there are cases where one of them works better but not other or > > >> vice-versa. You may utilise all three of them by having 3 different > > fields > > >> (fieldTypes) and during query, search in all of them. > > >> > > >> For some of the cases where none of them works (e.g wolves, wolf > etc)., > > use > > >> StemOverriderFactory. > > >> > > >> HTH. > > >> > > >> Thanks, > > >> Susheel > > >> > > >> On Thu, Dec 15, 2016 at 11:32 AM, Ahmet Arslan > > <iori...@yahoo.com.invalid> > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > KStemFilter returns legitimate English words, please use it. > > >> > > > >> > Ahmet > > >> > > > >> > > > >> > > > >> > On Thursday, December 15, 2016 6:17 PM, Lasitha Wattaladeniya < > > >> > watt...@gmail.com> wrote: > > >> > Hello devs, > > >> > > > >> > I'm trying to develop this indexing and querying flow where it > > converts > > >> the > > >> > words to its original form (lemmatization). I was doing bit of > > research > > >> > lately but the information on the internet is very limited. I tried > > using > > >> > hunspellfactory but it doesn't convert the word to it's original > form, > > >> > instead it gives suggestions for some words (hunspell works for some > > >> > english words correctly but for some it gives multiple suggestions > or > > no > > >> > suggestions, i used the en_us.dic provided by openoffice) > > >> > > > >> > I know this is a generic problem in searching, so is there anyone > who > > can > > >> > point me to correct direction or some information :) > > >> > > > >> > Best regards, > > >> > Lasitha Wattaladeniya > > >> > Software Engineer > > >> > > > >> > Mobile : +6593896893 > > >> > Blog : techreadme.blogspot.com > > >> > > > >> > > >
Re: Soft commit and reading data just after the commit
Hi Furkan,
Thanks for the links. I had read the first one but not the second one. I
did read it after you sent. So in my current solrconfig.xml settings below
are the configurations,
${solr.autoSoftCommit.maxTime:1}
15000
false
The problem i'm facing is, just after adding the documents to solr using
solrj, when I retrieve data from solr I am not getting the updated results.
This happens time to time. Most of the time I get the correct data but in
some occasions I get wrong results. so as you suggest, what the best
practice to use here ? , should I wait 1 mili second before calling for
updated results ?
Regards,
Lasitha
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893
Blog : techreadme.blogspot.com
On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com>
wrote:
> Hi Lasitha,
>
> First of all, did you check these:
>
> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
> https://lucidworks.com/blog/2013/08/23/understanding-
> transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> after that, if you cannot adjust your configuration you can give more
> information and we can find a solution.
>
> Kind Regards,
> Furkan KAMACI
>
> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <watt...@gmail.com>
> wrote:
>
>> Hi furkan,
>>
>> Thanks for your reply, it is generally a query heavy system. We are using
>> realtime indexing for editing the available data
>>
>> Regards,
>> Lasitha
>>
>> Lasitha Wattaladeniya
>> Software Engineer
>>
>> Mobile : +6593896893 <+65%209389%206893>
>> Blog : techreadme.blogspot.com
>>
>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <furkankam...@gmail.com>
>> wrote:
>>
>>> Hi Lasitha,
>>>
>>> What is your indexing / querying requirements. Do you have an index
>>> heavy/light - query heavy/light system?
>>>
>>> Kind Regards,
>>> Furkan KAMACI
>>>
>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>>> watt...@gmail.com>
>>> wrote:
>>>
>>> > Hello devs,
>>> >
>>> > I'm here with another problem i'm facing. I'm trying to do a commit
>>> (soft
>>> > commit) through solrj and just after the commit, retrieve the data from
>>> > solr (requirement is to get updated data list).
>>> >
>>> > I'm using soft commit instead of the hard commit, is previously I got
>>> an
>>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
>>> because of
>>> > too many commit requests. Now I have removed the explicit commit and
>>> has
>>> > let the solr to do the commit using autoSoftCommit *(1 mili second)*
>>> and
>>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
>>> errors
>>> > when i'm committing frequently.
>>> >
>>> > The problem i'm facing now is, I'm not getting the updated data when I
>>> > fetch from solr just after the soft commit. So in this case what are
>>> the
>>> > best practices to use ? to wait 1 mili second before retrieving data
>>> after
>>> > soft commit ? I don't feel like waiting from client side is a good
>>> option.
>>> > Please give me some help from your expert knowledge
>>> >
>>> > Best regards,
>>> > Lasitha Wattaladeniya
>>> > Software Engineer
>>> >
>>> > Mobile : +6593896893
>>> > Blog : techreadme.blogspot.com
>>> >
>>>
>>
>>
>
Re: Soft commit and reading data just after the commit
Hi furkan, Thanks for your reply, it is generally a query heavy system. We are using realtime indexing for editing the available data Regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <furkankam...@gmail.com> wrote: > Hi Lasitha, > > What is your indexing / querying requirements. Do you have an index > heavy/light - query heavy/light system? > > Kind Regards, > Furkan KAMACI > > On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <watt...@gmail.com > > > wrote: > > > Hello devs, > > > > I'm here with another problem i'm facing. I'm trying to do a commit (soft > > commit) through solrj and just after the commit, retrieve the data from > > solr (requirement is to get updated data list). > > > > I'm using soft commit instead of the hard commit, is previously I got an > > error "Exceeded limit of maxWarmingSearchers=2, try again later" because > of > > too many commit requests. Now I have removed the explicit commit and has > > let the solr to do the commit using autoSoftCommit *(1 mili second)* and > > autoCommit *(30 seconds)* configurations. Now I'm not getting any errors > > when i'm committing frequently. > > > > The problem i'm facing now is, I'm not getting the updated data when I > > fetch from solr just after the soft commit. So in this case what are the > > best practices to use ? to wait 1 mili second before retrieving data > after > > soft commit ? I don't feel like waiting from client side is a good > option. > > Please give me some help from your expert knowledge > > > > Best regards, > > Lasitha Wattaladeniya > > Software Engineer > > > > Mobile : +6593896893 > > Blog : techreadme.blogspot.com > > >
Soft commit and reading data just after the commit
Hello devs, I'm here with another problem i'm facing. I'm trying to do a commit (soft commit) through solrj and just after the commit, retrieve the data from solr (requirement is to get updated data list). I'm using soft commit instead of the hard commit, is previously I got an error "Exceeded limit of maxWarmingSearchers=2, try again later" because of too many commit requests. Now I have removed the explicit commit and has let the solr to do the commit using autoSoftCommit *(1 mili second)* and autoCommit *(30 seconds)* configurations. Now I'm not getting any errors when i'm committing frequently. The problem i'm facing now is, I'm not getting the updated data when I fetch from solr just after the soft commit. So in this case what are the best practices to use ? to wait 1 mili second before retrieving data after soft commit ? I don't feel like waiting from client side is a good option. Please give me some help from your expert knowledge Best regards, Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com
Re: Stemming with SOLR
Hi all, Thanks for the replies, @eric, ahmet : since those stemmers are logical stemmers it won't work on words such as caught, ran and so on. So in our case it won't work @susheel : Yes I thought about it but problems we have is, the documents we index are some what large text, so copy fielding these into duplicate fields will affect on the index time ( we have jobs to index data periodically) and query time. I wonder why there isn't a correct solution to this Regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Fri, Dec 16, 2016 at 12:58 AM, Susheel Kumar <susheel2...@gmail.com> wrote: > We did extensive comparison in the past for Snowball, KStem and Hunspell > and there are cases where one of them works better but not other or > vice-versa. You may utilise all three of them by having 3 different fields > (fieldTypes) and during query, search in all of them. > > For some of the cases where none of them works (e.g wolves, wolf etc)., use > StemOverriderFactory. > > HTH. > > Thanks, > Susheel > > On Thu, Dec 15, 2016 at 11:32 AM, Ahmet Arslan <iori...@yahoo.com.invalid> > wrote: > > > Hi, > > > > KStemFilter returns legitimate English words, please use it. > > > > Ahmet > > > > > > > > On Thursday, December 15, 2016 6:17 PM, Lasitha Wattaladeniya < > > watt...@gmail.com> wrote: > > Hello devs, > > > > I'm trying to develop this indexing and querying flow where it converts > the > > words to its original form (lemmatization). I was doing bit of research > > lately but the information on the internet is very limited. I tried using > > hunspellfactory but it doesn't convert the word to it's original form, > > instead it gives suggestions for some words (hunspell works for some > > english words correctly but for some it gives multiple suggestions or no > > suggestions, i used the en_us.dic provided by openoffice) > > > > I know this is a generic problem in searching, so is there anyone who can > > point me to correct direction or some information :) > > > > Best regards, > > Lasitha Wattaladeniya > > Software Engineer > > > > Mobile : +6593896893 > > Blog : techreadme.blogspot.com > > >
Stemming with SOLR
Hello devs, I'm trying to develop this indexing and querying flow where it converts the words to its original form (lemmatization). I was doing bit of research lately but the information on the internet is very limited. I tried using hunspellfactory but it doesn't convert the word to it's original form, instead it gives suggestions for some words (hunspell works for some english words correctly but for some it gives multiple suggestions or no suggestions, i used the en_us.dic provided by openoffice) I know this is a generic problem in searching, so is there anyone who can point me to correct direction or some information :) Best regards, Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com
How to get suggestions with numbers?
Hi devs, Past couple of days I was trying to get solr suggestions work with my applications. The requirement is as follows, I have some codes eg :abcdef2. I should be able to type "2000" and get suggestions as abcdef2. I index the field containing the code to a ngram field (standard analysers, nothing fancy). But it does not show the required suggestion. I think since I'm using spellchecker component with suggester, when I enter numbers the solr thinks I don't need suggestions. Can anybody help me to solve this problem? Regards, Lasitha
Re: Dynamically change solr suggest field
Hi Alessandro, Yes, using suggester is the correct way of doing. But in our scenario we thought of going with the spellchecker component since we had some legacy setup. Our plan is to move to suggester later on. So far spellchecker component also does the needed work for us. Regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Tue, May 17, 2016 at 2:15 AM, Alessandro Benedetti < benedetti.ale...@gmail.com> wrote: > The scenario you described should be done with the suggester component. > Nothing prevent you to configure multiple dictionaries for the suggester as > well. > The you pass the dictionary to the suggester at query time as a request > parameter for your suggester request handler. > > Cheers > On 16 May 2016 4:29 pm, "Abdel Belkasri" <belka...@gmail.com> wrote: > > > Clever and real cool. > > --Abdel > > > > On Sun, May 15, 2016 at 10:42 AM, Lasitha Wattaladeniya < > watt...@gmail.com > > > > > wrote: > > > > > Hello all, > > > > > > I found a way of doing this and thought of sharing this info with you. > I > > > found a way to dynamically change the field which gives the > suggestions. > > > It's using the solr spellchecker (Not suggester). You can basically > > > configure a indexed field as default *spellcheck.dictionary* in the > > config > > > file. Later you can set what ever the field you want suggestions from > in > > > the request (Can set more than 1) as the spellcheck.dictionary. This > way > > > you can set even multiple fields as spellchecker dictionaries and > > > suggestions will be returned according to the indexed values of those > > > field. > > > > > > Regards, > > > Lasitha. > > > > > > Lasitha Wattaladeniya > > > Software Engineer > > > > > > Mobile : +6593896893 > > > Blog : techreadme.blogspot.com > > > > > > On Thu, May 12, 2016 at 8:05 AM, Lasitha Wattaladeniya < > > watt...@gmail.com> > > > wrote: > > > > > > > Hi Nick, > > > > > > > > Thanks for the reply. According to my requirement I can use only > option > > > > one. I thought about that solution but I was bit lazy to implement > that > > > > since I have many modules and solr cores. If I'm going to configure > > > request > > > > handlers for each drop down value in each component it seems like a > lot > > > of > > > > work. Anyway this seems like the only way forward. > > > > > > > > I can't use the option two because the combo box select the filed, > not > > a > > > > value specific to a single field > > > > > > > > Best regards, > > > > Lasitha > > > > > > > > Lasitha Wattaladeniya > > > > Software Engineer > > > > > > > > Mobile : +6593896893 > > > > Blog : techreadme.blogspot.com > > > > > > > > On Wed, May 11, 2016 at 11:41 PM, Nick D <ndrake0...@gmail.com> > wrote: > > > > > > > >> There are only two ways I can think of to accomplish this and > neither > > of > > > >> them are dynamically setting the suggester field as is looks > according > > > to > > > >> the Doc (which does sometimes have lacking info so I might be wrong) > > you > > > >> cannot set something like *suggest.fl=combo_box_field* at query > time. > > > But > > > >> maybe they can help you get started. > > > >> > > > >> 1. Multiple suggester request handlers for each option in combo box. > > > This > > > >> way you just change the request handler in the query you submit > based > > on > > > >> the context. > > > >> > > > >> 2. Use copy fields to put all possible suggestions into same field > > name, > > > >> so > > > >> no more dynamic field settings, with another field defining whatever > > the > > > >> option would be for that document out of the combo box and use > context > > > >> filters which can be passed at query time to limit the suggestions > to > > > >> those > > > >> filtered by whats in the combo box. > > > >> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/solr/Suggester#Suggester-ContextFiltering > > > >> > > >
Re: Dynamically change solr suggest field
Hello all, I found a way of doing this and thought of sharing this info with you. I found a way to dynamically change the field which gives the suggestions. It's using the solr spellchecker (Not suggester). You can basically configure a indexed field as default *spellcheck.dictionary* in the config file. Later you can set what ever the field you want suggestions from in the request (Can set more than 1) as the spellcheck.dictionary. This way you can set even multiple fields as spellchecker dictionaries and suggestions will be returned according to the indexed values of those field. Regards, Lasitha. Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Thu, May 12, 2016 at 8:05 AM, Lasitha Wattaladeniya <watt...@gmail.com> wrote: > Hi Nick, > > Thanks for the reply. According to my requirement I can use only option > one. I thought about that solution but I was bit lazy to implement that > since I have many modules and solr cores. If I'm going to configure request > handlers for each drop down value in each component it seems like a lot of > work. Anyway this seems like the only way forward. > > I can't use the option two because the combo box select the filed, not a > value specific to a single field > > Best regards, > Lasitha > > Lasitha Wattaladeniya > Software Engineer > > Mobile : +6593896893 > Blog : techreadme.blogspot.com > > On Wed, May 11, 2016 at 11:41 PM, Nick D <ndrake0...@gmail.com> wrote: > >> There are only two ways I can think of to accomplish this and neither of >> them are dynamically setting the suggester field as is looks according to >> the Doc (which does sometimes have lacking info so I might be wrong) you >> cannot set something like *suggest.fl=combo_box_field* at query time. But >> maybe they can help you get started. >> >> 1. Multiple suggester request handlers for each option in combo box. This >> way you just change the request handler in the query you submit based on >> the context. >> >> 2. Use copy fields to put all possible suggestions into same field name, >> so >> no more dynamic field settings, with another field defining whatever the >> option would be for that document out of the combo box and use context >> filters which can be passed at query time to limit the suggestions to >> those >> filtered by whats in the combo box. >> >> https://cwiki.apache.org/confluence/display/solr/Suggester#Suggester-ContextFiltering >> >> Hope this helps a bit >> >> Nick >> >> On Wed, May 11, 2016 at 7:05 AM, Lasitha Wattaladeniya <watt...@gmail.com >> > >> wrote: >> >> > Hello devs, >> > >> > I'm trying to implement auto complete text suggestions using solr. I >> have a >> > text box and next to that there's a combo box. So the auto complete >> should >> > suggest based on the value selected in the combo box. >> > >> > Basically I should be able to change the suggest field based on the >> value >> > selected in the combo box. I was trying to solve this problem whole day >> but >> > not much luck. Can anybody tell me is there a way of doing this ? >> > >> > Regards, >> > Lasitha. >> > >> > Lasitha Wattaladeniya >> > Software Engineer >> > >> > Mobile : +6593896893 >> > Blog : techreadme.blogspot.com >> > >> > >
Re: Dynamically change solr suggest field
Hi Nick, Thanks for the reply. According to my requirement I can use only option one. I thought about that solution but I was bit lazy to implement that since I have many modules and solr cores. If I'm going to configure request handlers for each drop down value in each component it seems like a lot of work. Anyway this seems like the only way forward. I can't use the option two because the combo box select the filed, not a value specific to a single field Best regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Wed, May 11, 2016 at 11:41 PM, Nick D <ndrake0...@gmail.com> wrote: > There are only two ways I can think of to accomplish this and neither of > them are dynamically setting the suggester field as is looks according to > the Doc (which does sometimes have lacking info so I might be wrong) you > cannot set something like *suggest.fl=combo_box_field* at query time. But > maybe they can help you get started. > > 1. Multiple suggester request handlers for each option in combo box. This > way you just change the request handler in the query you submit based on > the context. > > 2. Use copy fields to put all possible suggestions into same field name, so > no more dynamic field settings, with another field defining whatever the > option would be for that document out of the combo box and use context > filters which can be passed at query time to limit the suggestions to those > filtered by whats in the combo box. > > https://cwiki.apache.org/confluence/display/solr/Suggester#Suggester-ContextFiltering > > Hope this helps a bit > > Nick > > On Wed, May 11, 2016 at 7:05 AM, Lasitha Wattaladeniya <watt...@gmail.com> > wrote: > > > Hello devs, > > > > I'm trying to implement auto complete text suggestions using solr. I > have a > > text box and next to that there's a combo box. So the auto complete > should > > suggest based on the value selected in the combo box. > > > > Basically I should be able to change the suggest field based on the value > > selected in the combo box. I was trying to solve this problem whole day > but > > not much luck. Can anybody tell me is there a way of doing this ? > > > > Regards, > > Lasitha. > > > > Lasitha Wattaladeniya > > Software Engineer > > > > Mobile : +6593896893 > > Blog : techreadme.blogspot.com > > >
Re: How to search string
Hi Kishor,
You can try escaping the search phrase "Garmin Class A" > Garmin\ Class\ A
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893
Blog : techreadme.blogspot.com
On Wed, May 11, 2016 at 6:12 PM, Ahmet Arslan <iori...@yahoo.com.invalid>
wrote:
> Hi,
>
> You can be explicit about the field that you want to search on. e.g.
> q=product_name:(Garmin Class A)
>
> Or you can use lucene query parser with default field (df) parameter. e.g.
> q={!lucene df=product_name)Garmin Class A
>
> Its all about query parsers.
>
> Ahmet
>
>
> On Wednesday, May 11, 2016 9:12 AM, kishor <krajus...@gmail.com> wrote:
> I want to search a product and product name is "Garmin Class A" so I
> expect
> result is product name matching string "Garmin Class A" but it searches
> separately i dont know why and how it happen.Please guide me how to search
> a
> string in only one field only not in other fields. "debug": {
> "rawquerystring": "Garmin Class A","querystring": "Garmin Class A",
> "parsedquery": "(+(DisjunctionMaxQuery((product_name:Garmin))
> DisjunctionMaxQuery((product_name:Class))
> DisjunctionMaxQuery((product_name:A))) ())/no_coord",
> "parsedquery_toString": "+((product_name:Garmin) (product_name:Class)
> (product_name:A)) ()","explain": {},
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/How-to-search-string-tp4276052.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Dynamically change solr suggest field
Hello devs, I'm trying to implement auto complete text suggestions using solr. I have a text box and next to that there's a combo box. So the auto complete should suggest based on the value selected in the combo box. Basically I should be able to change the suggest field based on the value selected in the combo box. I was trying to solve this problem whole day but not much luck. Can anybody tell me is there a way of doing this ? Regards, Lasitha. Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com
Re: Implementing partial search and exact matching
Hi Nick, Thanks for the reply. Actually q="software engineering" - > doc1 q="software engineer" - > no results q="Software engineer" - > doc2 I hope above test cases will explain my requirements further. So far I'm thinking of changing the qf according to query has enclosing double quotations or not. If somebody knows a better approach/queryparser please point me Thanks On 6 May 2016 10:56 am, "ND" <ndrake0...@gmail.com> wrote: > Lasitha, > > I think I understand what you are asking and if you have something like > Doc1 = software engineering > Doc2 = Software engineer > > And if you query > q=software+engineer -> Doc1 & Doc2 > > but > > q="software+engineer" -> Doc1 > > Correct? > > If this is correct then to my knowledge no, Solr out of the box cannot do > what you are asking, that is recognize a exact (quoted) search and change > the query fields to non-ngram fields. Of course you could do this in code > with some regex to see if the first and last character are double quotes > but there are a number of draw backs to this like q= "Software Engineer" > and Ninja wont do what you want it to but you could always do something > else. > > From my experience I would still want q="software+engineer" -> Doc1 and > Doc2 cause technically that exact phrase does exists in both Docs. > > Maybe there is someone else on here who can offer some more perspective on > this or a possible query analyzer that I haven't heard of that can solve > this issue (I would also be interested in that). > > Nick > > On Thu, May 5, 2016 at 6:33 PM, Lasitha Wattaladeniya <watt...@gmail.com> > wrote: > > > Hi nd, > > > > Here's the issue.. Let's say I search.. Software Engineer. For am > example > > lets say this query will return 10 results when search against ngram > field > > . Now I search "Software Engineer" with double quotations. This should > > not return same result set as the previous query. > > > > I thought the query parser I'm using (edismax) may have an inbuilt > > function for that. > > > > Do I have to specifically change the query field (qf) to solve this > issue > > for each query? > > On 6 May 2016 8:26 am, "ND" <ndrake0...@gmail.com> wrote: > > > > We implemented something similar it sounds to what you are asking but I > > dont see why you would need to match the original field. Since > technically > > a field that has *software engineer* indexed is matched by a query like > > "software eng" to "software engineer" with the ngrams; which makes > the > > exact phrase is still valid. > > > > The problem we where trying to solve was the exact phrase issue which can > > be solved with taking in a qs value of 0 (or higher depending on your > > definition of exactness). > > > > Maybe an example would help if there is something I am not understanding. > > > > Also your field definitions might help or a example of the schema > breakdown > > similar to the admin analyze page. > > > > Nick > > > > > > On Thu, May 5, 2016 at 12:25 AM, Lasitha Wattaladeniya < > watt...@gmail.com> > > wrote: > > > > > Hi All, > > > > > > I'm trying to implement a search functionality using solr. Currently > I'm > > > suing edismax parser with ngram fields to do the search against. So far > > it > > > works well. > > > > > > The question I have is when the user input double quotations to the > > search, > > > As the requirement this should match against the original field, not > > > against the ngram field. > > > > > > Currently what I have thought of doing is, identify the double > quotations > > > in the user input and change the query field (qf) according to that (to > > > ngram field or to the exact field). Isn't there any out of the box > > solution > > > for this, I feel like it's a generic requirement and don't want to > > reinvent > > > the wheel. Appreciate your comments > > > > > > [1]. https://issues.apache.org/jira/browse/SOLR-6842 > > > > > > [2]. > > > > > > > > > > > http://grokbase.com/t/lucene/solr-user/14cbghncvh/different-fields-for-user-supplied-phrases-in-edismax > > > > > > Thanks, > > > > > > Lasitha Wattaladeniya > > > > > > Software Engineer > > > > > >
Re: Implementing partial search and exact matching
Hi nd, Here's the issue.. Let's say I search.. Software Engineer. For am example lets say this query will return 10 results when search against ngram field . Now I search "Software Engineer" with double quotations. This should not return same result set as the previous query. I thought the query parser I'm using (edismax) may have an inbuilt function for that. Do I have to specifically change the query field (qf) to solve this issue for each query? On 6 May 2016 8:26 am, "ND" <ndrake0...@gmail.com> wrote: We implemented something similar it sounds to what you are asking but I dont see why you would need to match the original field. Since technically a field that has *software engineer* indexed is matched by a query like "software eng" to "software engineer" with the ngrams; which makes the exact phrase is still valid. The problem we where trying to solve was the exact phrase issue which can be solved with taking in a qs value of 0 (or higher depending on your definition of exactness). Maybe an example would help if there is something I am not understanding. Also your field definitions might help or a example of the schema breakdown similar to the admin analyze page. Nick On Thu, May 5, 2016 at 12:25 AM, Lasitha Wattaladeniya <watt...@gmail.com> wrote: > Hi All, > > I'm trying to implement a search functionality using solr. Currently I'm > suing edismax parser with ngram fields to do the search against. So far it > works well. > > The question I have is when the user input double quotations to the search, > As the requirement this should match against the original field, not > against the ngram field. > > Currently what I have thought of doing is, identify the double quotations > in the user input and change the query field (qf) according to that (to > ngram field or to the exact field). Isn't there any out of the box solution > for this, I feel like it's a generic requirement and don't want to reinvent > the wheel. Appreciate your comments > > [1]. https://issues.apache.org/jira/browse/SOLR-6842 > > [2]. > > http://grokbase.com/t/lucene/solr-user/14cbghncvh/different-fields-for-user-supplied-phrases-in-edismax > > Thanks, > > Lasitha Wattaladeniya > > Software Engineer >
Implementing partial search and exact search
Hi All, I'm trying to implement a search functionality using solr. Currently I'm suing edismax parser with ngram fields to do the search against. So far it works well. The question I have is when the user input double quotations to the search, As the requirement this should match against the original field, not against the ngram field. Currently what I have thought of doing is, identify the double quotations in the user input and change the query field (qf) according to that (to ngram field or to the exact field). Isn't there any out of the box solution for this, I feel like it's a generic requirement and don't want to reinvent the wheel. Appreciate your comments [1]. https://issues.apache.org/jira/browse/SOLR-6842 [2]. http://grokbase.com/t/lucene/solr-user/14cbghncvh/different-fields-for-user-supplied-phrases-in-edismax Thanks, Lasitha Wattaladeniya Software Engineer
Implementing partial search and exact matching
Hi All, I'm trying to implement a search functionality using solr. Currently I'm suing edismax parser with ngram fields to do the search against. So far it works well. The question I have is when the user input double quotations to the search, As the requirement this should match against the original field, not against the ngram field. Currently what I have thought of doing is, identify the double quotations in the user input and change the query field (qf) according to that (to ngram field or to the exact field). Isn't there any out of the box solution for this, I feel like it's a generic requirement and don't want to reinvent the wheel. Appreciate your comments [1]. https://issues.apache.org/jira/browse/SOLR-6842 [2]. http://grokbase.com/t/lucene/solr-user/14cbghncvh/different-fields-for-user-supplied-phrases-in-edismax Thanks, Lasitha Wattaladeniya Software Engineer
