RE: Handling Locales in Solr

2021-02-25 Thread Krönert Florian 
Hi Markus,

thank you a lot for your response, that helps us very much.

We will try out the approach of separating the cores by topic only.

Kind Regards,
Florian 

-Original Message-
From: Markus Jelsma  
Sent: Mittwoch, 24. Februar 2021 12:27
To: solr-user@lucene.apache.org
Subject: Re: Handling Locales in Solr

Hello,

We put all our customers in the same core/collection because of this, it is not 
practical to manage hundreds of cores, including their small overhead.
Although it can be advantageous when it comes to relevance tuning, no skewed 
statistics because of other customers.

In your case, an unused core is probably slow because it is not in cached 
memory anymore, and/or it has to load from a slow drive.

With regards to the locales, i would probably separate the cores by topic only, 
and have different languages share the same collection/core.

Regards,
Markus



Op wo 24 feb. 2021 om 12:09 schreef Krönert Florian <
florian.kroen...@orbis.de>:

> Hi everyone,
>
>
>
> First up thanks for this group, I appreciate it very much for 
> exchanging opinions on how to use Solr.
>
>
>
> We built a Solr instance for one of our customers which is used for 
> searching data on his website.
>
> We need to search different data (kb articles, products and external
> links) in different locales.
>
>
>
> For our logic it seemed best to separate solr Cores by topic and 
> locale, so we have cores like this:
>
> kbarticle_de-de
>
> kbarticle_en-us
>
> …
>
> products_de-de
>
> products_en-us
>
> …
>
> links_de-de
>
> links_en-us
>
>
>
> First we had only 3 locales, but it grew pretty fast to 16 locales, so 
> that we’re having 48 solr cores by now already.
>
> There would have been different approaches for realizing this of 
> course, so we’re wondering whether we are using Solr not in the optimal way?
>
>
>
> We found out that when a search on a locale that was not used for some 
> time is started, it takes >10 seconds in many cases to execute the search.
>
>
>
> We then find logs like this, where it seems as if Solr needs to start 
> a searcher first, which takes time:
>
> 2021-02-20 04:33:42.634 INFO  (Thread-20674) [   ]
> o.a.s.s.SolrIndexSearcher Opening [Searcher@775f8595[kbarticles_en-gb]
> main]
>
> 2021-02-20 04:33:42.643 INFO  (searcherExecutor-26-thread-1) [   ]
> o.a.s.c.QuerySenderListener QuerySenderListener sending requests to 
> Searcher@775f8595[kbarticles_en-gb]
>
> …
>
>
>
> Is that an issue? It would be good to know whether our localization 
> approach causes issues with Solr and whether we should restructure our 
> core design.
>
> Any help would be very much appreciated.
>
>
>
> Kind Regards,
>
>
>
> *Florian Krönert*
> Senior Software Developer
>
> <https://www.orbis.de/de.html>
>
> *ORBIS AG | *Planckstraße 10 | D-88677 Markdorf
>
> Phone: +49 7544 50398 21 | Mobile: +49 162 3065972 | E-Mail:
> florian.kroen...@orbis.de
> www.orbis.de
>
>
> <https://www.orbis.de/de/sap-by-orbis.html>
>
> Registered Seat: Saarbrücken
> Commercial Register Court: Amtsgericht Saarbrücken, HRB 12022 Board of 
> Management: Thomas Gard (Chairman), Michael Jung, Stefan Mailänder, 
> Frank Schmelzer Chairman of the Supervisory Board: Ulrich Holzer
>
> <https://www.facebook.com/ORBIS.AG>
> <https://www.linkedin.com/company/orbis-ag/>
> <https://www.xing.com/companies/orbisag><https://twitter.com/ORBIS_AG>
> 
> <https://www.youtube.com/channel/UC5Fx5UsUy4FkzB0dfCWXwYw/featured>
>
>
>
>
> <https://www.orbis.de/de/aktuelles/news/aktuelle-presse/news-detail/ar
> ticle/inner-circle-award-orbis-zaehlt-erneut-zu-den-weltbesten-partner
> n-fuer-microsoft-business-application.html>
>
>
>
>
> <https://www.orbis.de/de/microsoft-by-orbis/beratungskompetenz/power-p
> latform.html?wmc=Banner-Microsoft-Power-Platform>
>
>
> <https://www.orbis.de/de/microsoft-by-orbis.html?wmc=Banner-Microsoft-
> by-ORBIS>
>

Re: Handling Locales in Solr

2021-02-24 Thread Markus Jelsma 
Hello,

We put all our customers in the same core/collection because of this, it is
not practical to manage hundreds of cores, including their small overhead.
Although it can be advantageous when it comes to relevance tuning, no
skewed statistics because of other customers.

In your case, an unused core is probably slow because it is not in cached
memory anymore, and/or it has to load from a slow drive.

With regards to the locales, i would probably separate the cores by topic
only, and have different languages share the same collection/core.

Regards,
Markus



Op wo 24 feb. 2021 om 12:09 schreef Krönert Florian <
florian.kroen...@orbis.de>:

> Hi everyone,
>
>
>
> First up thanks for this group, I appreciate it very much for exchanging
> opinions on how to use Solr.
>
>
>
> We built a Solr instance for one of our customers which is used for
> searching data on his website.
>
> We need to search different data (kb articles, products and external
> links) in different locales.
>
>
>
> For our logic it seemed best to separate solr Cores by topic and locale,
> so we have cores like this:
>
> kbarticle_de-de
>
> kbarticle_en-us
>
> …
>
> products_de-de
>
> products_en-us
>
> …
>
> links_de-de
>
> links_en-us
>
>
>
> First we had only 3 locales, but it grew pretty fast to 16 locales, so
> that we’re having 48 solr cores by now already.
>
> There would have been different approaches for realizing this of course,
> so we’re wondering whether we are using Solr not in the optimal way?
>
>
>
> We found out that when a search on a locale that was not used for some
> time is started, it takes >10 seconds in many cases to execute the search.
>
>
>
> We then find logs like this, where it seems as if Solr needs to start a
> searcher first, which takes time:
>
> 2021-02-20 04:33:42.634 INFO  (Thread-20674) [   ]
> o.a.s.s.SolrIndexSearcher Opening [Searcher@775f8595[kbarticles_en-gb]
> main]
>
> 2021-02-20 04:33:42.643 INFO  (searcherExecutor-26-thread-1) [   ]
> o.a.s.c.QuerySenderListener QuerySenderListener sending requests to
> Searcher@775f8595[kbarticles_en-gb]
>
> …
>
>
>
> Is that an issue? It would be good to know whether our localization
> approach causes issues with Solr and whether we should restructure our core
> design.
>
> Any help would be very much appreciated.
>
>
>
> Kind Regards,
>
>
>
> *Florian Krönert*
> Senior Software Developer
>
> 
>
> *ORBIS AG | *Planckstraße 10 | D-88677 Markdorf
>
> Phone: +49 7544 50398 21 | Mobile: +49 162 3065972 | E-Mail:
> florian.kroen...@orbis.de
> www.orbis.de
>
>
> 
>
> Registered Seat: Saarbrücken
> Commercial Register Court: Amtsgericht Saarbrücken, HRB 12022
> Board of Management: Thomas Gard (Chairman), Michael Jung, Stefan
> Mailänder, Frank Schmelzer
> Chairman of the Supervisory Board: Ulrich Holzer
>
> 
> 
> 
> 
>
>
>
>
> 
>
>
>
>
> 
>
>
> 
>

Handling Locales in Solr

2021-02-24 Thread Krönert Florian 
Hi everyone,

First up thanks for this group, I appreciate it very much for exchanging 
opinions on how to use Solr.

We built a Solr instance for one of our customers which is used for searching 
data on his website.
We need to search different data (kb articles, products and external links) in 
different locales.

For our logic it seemed best to separate solr Cores by topic and locale, so we 
have cores like this:
kbarticle_de-de
kbarticle_en-us
...
products_de-de
products_en-us
...
links_de-de
links_en-us

First we had only 3 locales, but it grew pretty fast to 16 locales, so that 
we're having 48 solr cores by now already.
There would have been different approaches for realizing this of course, so 
we're wondering whether we are using Solr not in the optimal way?

We found out that when a search on a locale that was not used for some time is 
started, it takes >10 seconds in many cases to execute the search.

We then find logs like this, where it seems as if Solr needs to start a 
searcher first, which takes time:
2021-02-20 04:33:42.634 INFO  (Thread-20674) [   ] o.a.s.s.SolrIndexSearcher 
Opening [Searcher@775f8595[kbarticles_en-gb] main]
2021-02-20 04:33:42.643 INFO  (searcherExecutor-26-thread-1) [   ] 
o.a.s.c.QuerySenderListener QuerySenderListener sending requests to 
Searcher@775f8595[kbarticles_en-gb]
...

Is that an issue? It would be good to know whether our localization approach 
causes issues with Solr and whether we should restructure our core design.
Any help would be very much appreciated.

Kind Regards,

Florian Krönert
Senior Software Developer

[cid:image001.gif@01D70AA4.14778350]
ORBIS AG | Planckstraße 10 | D-88677 Markdorf
Phone: +49 7544 50398 21 | Mobile: +49 162 3065972 | E-Mail: 
florian.kroen...@orbis.de
www.orbis.de

[cid:image002.png@01D70AA4.14778350] 
   [cid:image003.jpg@01D70AA4.14778350]

Registered Seat: Saarbrücken
Commercial Register Court: Amtsgericht Saarbrücken, HRB 12022
Board of Management: Thomas Gard (Chairman), Michael Jung, Stefan Mailänder, 
Frank Schmelzer
Chairman of the Supervisory Board: Ulrich Holzer
[cid:image004.png@01D70AA4.14778350]   
[cid:image005.png@01D70AA4.14778350] 

[cid:image006.png@01D70AA4.14778350]    
 [cid:image007.png@01D70AA4.14778350] 
[cid:image008.png@01D70AA4.14778350] 

[cid:image009.png@01D70AA4.14778350]

[cid:image010.png@01D70AA4.14778350]


[cid:MicrosoftPowerPlatform_7054080f-b0bf-4e97-b9ab-55dae1373165.png]

[cid:Microsoftallgemein_e6028b0b-75cc-43da-9046-9fec28faad6d.png]