I would also suggest you look at which GC mechanism you use. Increasing RAM and Heap-Size might result in the application freezed for a long time (during GC).
Deepak "The greatness of a nation can be judged by the way its animals are treated - Mahatma Gandhi" +91 73500 12833 deic...@gmail.com Facebook: https://www.facebook.com/deicool LinkedIn: www.linkedin.com/in/deicool "Plant a Tree, Go Green" Make In India : http://www.makeinindia.com/home On Tue, Jul 5, 2022 at 4:43 PM Dave <hastings.recurs...@gmail.com> wrote: > Exactly. You could have the best engineer on your continent but the end > result is the same, more metal. I could build a very fast search server > for less than a week of my salary so what’s the point of wasting two weeks > trying to solve a problem when the solution is literally just right there, > a big ssd and a lot of memory, then I could work on things that are > actually important rather than try to squeeze blood from a turnip. > > > On Jul 5, 2022, at 6:11 AM, Charlie Hull < > ch...@opensourceconnections.com> wrote: > > > > I think you're missing my point. > > > > Good engineers, even mediocre ones, are expensive, and the great ones > are rare. It's a tedious task chasing tiny performance gains when you know > you're limited by the hardware and a bored engineer might just go and look > for another job. So if you fail to realise that a capital expense for > hardware or hosting is necessary, you run the risk of losing the people > that make your search engine work (even if they stay they could also be > doing something more useful to your business). > > > > Charlie > > > >> On 05/07/2022 10:49, Deepak Goel wrote: > >> If you are tearing your hair out on 'Number of Hours' required for > tuning > >> your software, it's time you switch to a better quality performance > >> engineer. > >> > >> Deepak > >> "The greatness of a nation can be judged by the way its animals are > treated > >> - Mahatma Gandhi" > >> > >> +91 73500 12833 > >> deic...@gmail.com > >> > >> Facebook:https://www.facebook.com/deicool > >> LinkedIn:www.linkedin.com/in/deicool > >> > >> "Plant a Tree, Go Green" > >> > >> Make In India :http://www.makeinindia.com/home > >> > >> > >> On Tue, Jul 5, 2022 at 3:12 PM Charlie Hull< > ch...@opensourceconnections.com> > >> wrote: > >> > >>> Equally it's not a good management practice to burn engineering hours > >>> trying to optimise performance to avoid spending (often much less) > money > >>> on sufficient hardware to do the job. I've seen this happen many times, > >>> sadly. > >>> > >>> Charlie > >>> > >>> On 05/07/2022 10:33, Deepak Goel wrote: > >>>> Not a good software engineering practice to beef up the hardware > blindly. > >>>> Of Course when you have tuned the software to a point where you can't > >>> tune > >>>> anymore, you can then turn your eyes to hardware. > >>>> > >>>> Deepak > >>>> "The greatness of a nation can be judged by the way its animals are > >>> treated > >>>> - Mahatma Gandhi" > >>>> > >>>> +91 73500 12833 > >>>> deic...@gmail.com > >>>> > >>>> Facebook:https://www.facebook.com/deicool > >>>> LinkedIn:www.linkedin.com/in/deicool > >>>> > >>>> "Plant a Tree, Go Green" > >>>> > >>>> Make In India :http://www.makeinindia.com/home > >>>> > >>>> > >>>> On Tue, Jul 5, 2022 at 1:01 AM Dave<hastings.recurs...@gmail.com> > >>> wrote: > >>>>> Also for $115 I can buy a terabyte of a Samsung ssd, which helps a > lot. > >>> It > >>>>> comes to a point where money on hardware will outweigh money on > >>> engineering > >>>>> man power hours, and still come to the same conclusion. As much ram > as > >>> your > >>>>> rack can take and as big and fast of a raid ssd drive it can take. > >>> Remember > >>>>> since solr is always meant to be destroyed and recreated you don’t > have > >>> to > >>>>> worry much about hardware failure if you just buy two of everything > and > >>>>> have a backup server ready and waiting to take over while the > original > >>>>> fails and is reconstructed. > >>>>> > >>>>>> On Jul 4, 2022, at 1:32 PM, Shawn Heisey<apa...@elyograg.org> > wrote: > >>>>>> > >>>>>> On 7/4/22 03:01, Mike wrote: > >>>>>>> My Solr index size is around 500GB and I have 64GB of RAM. Solr > eats > >>> up > >>>>> all > >>>>>>> the memory and because of that PHP works very, very slowly. What > can I > >>>>> do? > >>>>>> Solr is a Java program. A Java program will never directly use more > >>>>> memory than you specify for the max heap size. We cannot make any > >>> general > >>>>> recommendations about what heap size you need, because there is a > good > >>>>> chance that any recommendation we make would be completely wrong for > >>> your > >>>>> install. I did see that someone recommended not going above 31G ... > and > >>>>> this is good advice. At 32 GB, Java switches to 64-bit pointers > >>> instead of > >>>>> 32-bit. So a heap size of 32 GB actually has LESS memory available > >>> than a > >>>>> heap size of 31 GB. > >>>>>> The OS will use additional memory beyond the heap for caching the > index > >>>>> data, but that is completely outside of Solr's control. Note that > 64GB > >>>>> total memory for a 500GB index is almost certainly not enough memory, > >>>>> ESPECIALLY if the same server is used for things other than Solr. I > >>> wrote > >>>>> the following wiki page: > >>> > https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems > >>>>>> Others have recommended that you run Solr on dedicated hardware > that is > >>>>> not used for any other purpose. I concur with that recommendation. > >>>>>> Thanks, > >>>>>> Shawn > >>>>>> > >>> -- > >>> Charlie Hull - Managing Consultant at OpenSource Connections Limited > >>> Founding member of The Search Network<http://www.thesearchnetwork.com> > >>> and co-author of Searching the Enterprise > >>> < > >>> > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > >>> tel/fax: +44 (0)8700 118334 > >>> mobile: +44 (0)7767 825828 > >>> > >>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin > >>> Amtsgericht Charlottenburg | HRB 230712 B > >>> Geschäftsführer: John M. Woodell | David E. Pugh > >>> Finanzamt: Berlin Finanzamt für Körperschaften II > >>> > >>> -- > >>> This email has been checked for viruses by AVG. > >>> https://www.avg.com > >>> > > -- > > Charlie Hull - Managing Consultant at OpenSource Connections Limited > > Founding member of The Search Network <http://www.thesearchnetwork.com> > and co-author of Searching the Enterprise < > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > > > > tel/fax: +44 (0)8700 118334 > > mobile: +44 (0)7767 825828 > > > > OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin > > Amtsgericht Charlottenburg | HRB 230712 B > > Geschäftsführer: John M. Woodell | David E. Pugh > > Finanzamt: Berlin Finanzamt für Körperschaften II > > > > -- > > This email has been checked for viruses by AVG. > > https://www.avg.com >
