Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
I don't recall anyone being able to get acceptable performance with a single index that large with solr/lucene. The conventional wisdom is that parallel searching across cores (or shards in SolrCloud) is the best way to handle index sizes in the "illions". So its of great interest how you did. Anyone else gotten an index(es) with billions of documents to perform well? I'm greatly interested in how. On Mon, 2012-05-28 at 05:12 -0700, Nagendra Nagarajayya wrote: > It is a single node. I am trying to find out if the performance can be > referenced. > > Regarding information on Solr with RankingAlgorithm, you can find all > the information here: > > http://solr-ra.tgels.org > > On RankingAlgorithm: > > http://rankingalgorithm.tgels.org > > Regards, > - NN > > On 5/27/2012 4:50 PM, Li Li wrote: > > yes, I am also interested in good performance with 2 billion docs. how > > many search nodes do you use? what's the average response time and qps > > ? > > > > another question: where can I find related paper or resources of your > > algorithm which explains the algorithm in detail? why it's better than > > google site(better than lucene is not very interested because lucene > > is not originally designed to provide search function like google)? > > > > On Mon, May 28, 2012 at 1:06 AM, Darren Govoni wrote: > >> I think people on this list would be more interested in your approach to > >> scaling 2 billion documents than modifying solr/lucene scoring (which is > >> already top notch). So given that, can you share any references or > >> otherwise substantiate good performance with 2 billion documents? > >> > >> Thanks. > >> > >> On Sun, 2012-05-27 at 08:29 -0700, Nagendra Nagarajayya wrote: > >>> Actually, RankingAlgorithm 1.4.2 has been scaled to more than 2 billion > >>> docs. With RankingAlgorithm 1.4.3, using the parameters > >>> age=latest&docs=number feature, you can retrieve the NRT inserted > >>> documents in milliseconds from such a huge index improving query and > >>> faceting performance and using very little resources ... > >>> > >>> Currently, RankingAlgorithm 1.4.3 is only available with Solr 4.0, and > >>> the NRT insert performance with Solr 4.0 is about 70,000 docs / sec. > >>> RankingAlgorithm 1.4.3 should become available with Solr 3.6 soon. > >>> > >>> Regards, > >>> > >>> Nagendra Nagarajayya > >>> http://solr-ra.tgels.org > >>> http://rankingalgorithm.tgels.org > >>> > >>> > >>> > >>> On 5/27/2012 7:32 AM, Darren Govoni wrote: > Hi, > Have you tested this with a billion documents? > > Darren > > On Sun, 2012-05-27 at 07:24 -0700, Nagendra Nagarajayya wrote: > > Hi! > > > > I am very excited to announce the availability of Solr 3.6 with > > RankingAlgorithm 1.4.2. > > > > This NRT supports now works with both RankingAlgorithm and Lucene. The > > insert/update performance should be about 5000 docs in about 490 ms with > > the MbArtists Index. > > > > RankingAlgorithm 1.4.2 has multiple algorithms, improved performance > > over the earlier releases, supports the entire Lucene Query Syntax, ± > > and/or boolean queries and can scale to more than a billion documents. > > > > You can get more information about NRT performance from here: > > http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x > > > > You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: > > http://solr-ra.tgels.org > > > > Please download and give the new version a try. > > > > Regards, > > > > Nagendra Nagarajayya > > http://solr-ra.tgels.org > > http://rankingalgorithm.tgels.org > > > > ps. MbArtists index is the example index used in the Solr 1.4 Enterprise > > Book > > > > > >> > > >
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
It is a single node. I am trying to find out if the performance can be referenced. Regarding information on Solr with RankingAlgorithm, you can find all the information here: http://solr-ra.tgels.org On RankingAlgorithm: http://rankingalgorithm.tgels.org Regards, - NN On 5/27/2012 4:50 PM, Li Li wrote: yes, I am also interested in good performance with 2 billion docs. how many search nodes do you use? what's the average response time and qps ? another question: where can I find related paper or resources of your algorithm which explains the algorithm in detail? why it's better than google site(better than lucene is not very interested because lucene is not originally designed to provide search function like google)? On Mon, May 28, 2012 at 1:06 AM, Darren Govoni wrote: I think people on this list would be more interested in your approach to scaling 2 billion documents than modifying solr/lucene scoring (which is already top notch). So given that, can you share any references or otherwise substantiate good performance with 2 billion documents? Thanks. On Sun, 2012-05-27 at 08:29 -0700, Nagendra Nagarajayya wrote: Actually, RankingAlgorithm 1.4.2 has been scaled to more than 2 billion docs. With RankingAlgorithm 1.4.3, using the parameters age=latest&docs=number feature, you can retrieve the NRT inserted documents in milliseconds from such a huge index improving query and faceting performance and using very little resources ... Currently, RankingAlgorithm 1.4.3 is only available with Solr 4.0, and the NRT insert performance with Solr 4.0 is about 70,000 docs / sec. RankingAlgorithm 1.4.3 should become available with Solr 3.6 soon. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org On 5/27/2012 7:32 AM, Darren Govoni wrote: Hi, Have you tested this with a billion documents? Darren On Sun, 2012-05-27 at 07:24 -0700, Nagendra Nagarajayya wrote: Hi! I am very excited to announce the availability of Solr 3.6 with RankingAlgorithm 1.4.2. This NRT supports now works with both RankingAlgorithm and Lucene. The insert/update performance should be about 5000 docs in about 490 ms with the MbArtists Index. RankingAlgorithm 1.4.2 has multiple algorithms, improved performance over the earlier releases, supports the entire Lucene Query Syntax, ± and/or boolean queries and can scale to more than a billion documents. You can get more information about NRT performance from here: http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: http://solr-ra.tgels.org Please download and give the new version a try. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org ps. MbArtists index is the example index used in the Solr 1.4 Enterprise Book
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
yes, I am also interested in good performance with 2 billion docs. how many search nodes do you use? what's the average response time and qps ? another question: where can I find related paper or resources of your algorithm which explains the algorithm in detail? why it's better than google site(better than lucene is not very interested because lucene is not originally designed to provide search function like google)? On Mon, May 28, 2012 at 1:06 AM, Darren Govoni wrote: > I think people on this list would be more interested in your approach to > scaling 2 billion documents than modifying solr/lucene scoring (which is > already top notch). So given that, can you share any references or > otherwise substantiate good performance with 2 billion documents? > > Thanks. > > On Sun, 2012-05-27 at 08:29 -0700, Nagendra Nagarajayya wrote: >> Actually, RankingAlgorithm 1.4.2 has been scaled to more than 2 billion >> docs. With RankingAlgorithm 1.4.3, using the parameters >> age=latest&docs=number feature, you can retrieve the NRT inserted >> documents in milliseconds from such a huge index improving query and >> faceting performance and using very little resources ... >> >> Currently, RankingAlgorithm 1.4.3 is only available with Solr 4.0, and >> the NRT insert performance with Solr 4.0 is about 70,000 docs / sec. >> RankingAlgorithm 1.4.3 should become available with Solr 3.6 soon. >> >> Regards, >> >> Nagendra Nagarajayya >> http://solr-ra.tgels.org >> http://rankingalgorithm.tgels.org >> >> >> >> On 5/27/2012 7:32 AM, Darren Govoni wrote: >> > Hi, >> > Have you tested this with a billion documents? >> > >> > Darren >> > >> > On Sun, 2012-05-27 at 07:24 -0700, Nagendra Nagarajayya wrote: >> >> Hi! >> >> >> >> I am very excited to announce the availability of Solr 3.6 with >> >> RankingAlgorithm 1.4.2. >> >> >> >> This NRT supports now works with both RankingAlgorithm and Lucene. The >> >> insert/update performance should be about 5000 docs in about 490 ms with >> >> the MbArtists Index. >> >> >> >> RankingAlgorithm 1.4.2 has multiple algorithms, improved performance >> >> over the earlier releases, supports the entire Lucene Query Syntax, ± >> >> and/or boolean queries and can scale to more than a billion documents. >> >> >> >> You can get more information about NRT performance from here: >> >> http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x >> >> >> >> You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: >> >> http://solr-ra.tgels.org >> >> >> >> Please download and give the new version a try. >> >> >> >> Regards, >> >> >> >> Nagendra Nagarajayya >> >> http://solr-ra.tgels.org >> >> http://rankingalgorithm.tgels.org >> >> >> >> ps. MbArtists index is the example index used in the Solr 1.4 Enterprise >> >> Book >> >> >> > >> > >> > >> > >
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
Let me see if I can reference the performance with RankingAlgorithm. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org On 5/27/2012 10:06 AM, Darren Govoni wrote: I think people on this list would be more interested in your approach to scaling 2 billion documents than modifying solr/lucene scoring (which is already top notch). So given that, can you share any references or otherwise substantiate good performance with 2 billion documents? Thanks. On Sun, 2012-05-27 at 08:29 -0700, Nagendra Nagarajayya wrote: Actually, RankingAlgorithm 1.4.2 has been scaled to more than 2 billion docs. With RankingAlgorithm 1.4.3, using the parameters age=latest&docs=number feature, you can retrieve the NRT inserted documents in milliseconds from such a huge index improving query and faceting performance and using very little resources ... Currently, RankingAlgorithm 1.4.3 is only available with Solr 4.0, and the NRT insert performance with Solr 4.0 is about 70,000 docs / sec. RankingAlgorithm 1.4.3 should become available with Solr 3.6 soon. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org On 5/27/2012 7:32 AM, Darren Govoni wrote: Hi, Have you tested this with a billion documents? Darren On Sun, 2012-05-27 at 07:24 -0700, Nagendra Nagarajayya wrote: Hi! I am very excited to announce the availability of Solr 3.6 with RankingAlgorithm 1.4.2. This NRT supports now works with both RankingAlgorithm and Lucene. The insert/update performance should be about 5000 docs in about 490 ms with the MbArtists Index. RankingAlgorithm 1.4.2 has multiple algorithms, improved performance over the earlier releases, supports the entire Lucene Query Syntax, ± and/or boolean queries and can scale to more than a billion documents. You can get more information about NRT performance from here: http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: http://solr-ra.tgels.org Please download and give the new version a try. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org ps. MbArtists index is the example index used in the Solr 1.4 Enterprise Book
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
I think people on this list would be more interested in your approach to scaling 2 billion documents than modifying solr/lucene scoring (which is already top notch). So given that, can you share any references or otherwise substantiate good performance with 2 billion documents? Thanks. On Sun, 2012-05-27 at 08:29 -0700, Nagendra Nagarajayya wrote: > Actually, RankingAlgorithm 1.4.2 has been scaled to more than 2 billion > docs. With RankingAlgorithm 1.4.3, using the parameters > age=latest&docs=number feature, you can retrieve the NRT inserted > documents in milliseconds from such a huge index improving query and > faceting performance and using very little resources ... > > Currently, RankingAlgorithm 1.4.3 is only available with Solr 4.0, and > the NRT insert performance with Solr 4.0 is about 70,000 docs / sec. > RankingAlgorithm 1.4.3 should become available with Solr 3.6 soon. > > Regards, > > Nagendra Nagarajayya > http://solr-ra.tgels.org > http://rankingalgorithm.tgels.org > > > > On 5/27/2012 7:32 AM, Darren Govoni wrote: > > Hi, > >Have you tested this with a billion documents? > > > > Darren > > > > On Sun, 2012-05-27 at 07:24 -0700, Nagendra Nagarajayya wrote: > >> Hi! > >> > >> I am very excited to announce the availability of Solr 3.6 with > >> RankingAlgorithm 1.4.2. > >> > >> This NRT supports now works with both RankingAlgorithm and Lucene. The > >> insert/update performance should be about 5000 docs in about 490 ms with > >> the MbArtists Index. > >> > >> RankingAlgorithm 1.4.2 has multiple algorithms, improved performance > >> over the earlier releases, supports the entire Lucene Query Syntax, ± > >> and/or boolean queries and can scale to more than a billion documents. > >> > >> You can get more information about NRT performance from here: > >> http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x > >> > >> You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: > >> http://solr-ra.tgels.org > >> > >> Please download and give the new version a try. > >> > >> Regards, > >> > >> Nagendra Nagarajayya > >> http://solr-ra.tgels.org > >> http://rankingalgorithm.tgels.org > >> > >> ps. MbArtists index is the example index used in the Solr 1.4 Enterprise > >> Book > >> > > > > > > >
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
Solr with RankingAlgorithm is free (for test/development/production). Here is the license page http://tgels.com/docs/SoftwareLicenseAgreement.txt. RankingAlgorithm is also free ( for test/development/production). Regards, - NN On 5/27/2012 8:57 AM, Radim Kolar wrote: My company is thinking to buy search algorithm from famous expert in searching Petr Hejl - http://www.milionovastranka.net/ but i see RankingAlgorithm has fantastic results too and looking at its reference page it even powers sites like oracle.com and ebay.com.
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
On Sun, May 27, 2012 at 12:42 PM, Radim Kolar wrote: >> What reference page are you referring to? > > http://tgels.com/wiki/en/Sites_using/downloaded_RankingAlgorithm_or_Solr-RA Ah, ok "sites using/downloaded" So someone with a .oracle email / domain checked it out - that certainly doesn't mean they are in production with it, or even plan to be. -Yonik http://lucidimagination.com
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
What reference page are you referring to? http://tgels.com/wiki/en/Sites_using/downloaded_RankingAlgorithm_or_Solr-RA
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
On Sun, May 27, 2012 at 11:57 AM, Radim Kolar wrote: > but i see RankingAlgorithm has fantastic results too and looking at its > reference page it even powers sites like oracle.com and ebay.com. What reference page are you referring to? -Yonik http://lucidimagination.com
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
My company is thinking to buy search algorithm from famous expert in searching Petr Hejl - http://www.milionovastranka.net/ but i see RankingAlgorithm has fantastic results too and looking at its reference page it even powers sites like oracle.com and ebay.com.
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
Actually, RankingAlgorithm 1.4.2 has been scaled to more than 2 billion docs. With RankingAlgorithm 1.4.3, using the parameters age=latest&docs=number feature, you can retrieve the NRT inserted documents in milliseconds from such a huge index improving query and faceting performance and using very little resources ... Currently, RankingAlgorithm 1.4.3 is only available with Solr 4.0, and the NRT insert performance with Solr 4.0 is about 70,000 docs / sec. RankingAlgorithm 1.4.3 should become available with Solr 3.6 soon. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org On 5/27/2012 7:32 AM, Darren Govoni wrote: Hi, Have you tested this with a billion documents? Darren On Sun, 2012-05-27 at 07:24 -0700, Nagendra Nagarajayya wrote: Hi! I am very excited to announce the availability of Solr 3.6 with RankingAlgorithm 1.4.2. This NRT supports now works with both RankingAlgorithm and Lucene. The insert/update performance should be about 5000 docs in about 490 ms with the MbArtists Index. RankingAlgorithm 1.4.2 has multiple algorithms, improved performance over the earlier releases, supports the entire Lucene Query Syntax, ± and/or boolean queries and can scale to more than a billion documents. You can get more information about NRT performance from here: http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: http://solr-ra.tgels.org Please download and give the new version a try. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org ps. MbArtists index is the example index used in the Solr 1.4 Enterprise Book
Re: [Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
Hi, Have you tested this with a billion documents? Darren On Sun, 2012-05-27 at 07:24 -0700, Nagendra Nagarajayya wrote: > Hi! > > I am very excited to announce the availability of Solr 3.6 with > RankingAlgorithm 1.4.2. > > This NRT supports now works with both RankingAlgorithm and Lucene. The > insert/update performance should be about 5000 docs in about 490 ms with > the MbArtists Index. > > RankingAlgorithm 1.4.2 has multiple algorithms, improved performance > over the earlier releases, supports the entire Lucene Query Syntax, ± > and/or boolean queries and can scale to more than a billion documents. > > You can get more information about NRT performance from here: > http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x > > You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: > http://solr-ra.tgels.org > > Please download and give the new version a try. > > Regards, > > Nagendra Nagarajayya > http://solr-ra.tgels.org > http://rankingalgorithm.tgels.org > > ps. MbArtists index is the example index used in the Solr 1.4 Enterprise > Book >
[Announce] Solr 3.6 with RankingAlgorithm 1.4.2 - NRT support
Hi! I am very excited to announce the availability of Solr 3.6 with RankingAlgorithm 1.4.2. This NRT supports now works with both RankingAlgorithm and Lucene. The insert/update performance should be about 5000 docs in about 490 ms with the MbArtists Index. RankingAlgorithm 1.4.2 has multiple algorithms, improved performance over the earlier releases, supports the entire Lucene Query Syntax, ± and/or boolean queries and can scale to more than a billion documents. You can get more information about NRT performance from here: http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_3.x You can download Solr 3.6 with RankingAlgorithm 1.4.2 from here: http://solr-ra.tgels.org Please download and give the new version a try. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org ps. MbArtists index is the example index used in the Solr 1.4 Enterprise Book
