I was about to mention this too, as the ram prediction was way lower the one in reality. But I very much like the idea of the tool. Possibly, integrating into the admin UI at some point later would gain more light to it. On 21 Jun 2013 03:55, "Erick Erickson" <[email protected]> wrote:
> Gaaah, I've been looking at this more today and it's completely not > ready for prime-time. Numbers are bogus, so don't rely on it much. > > On Thu, Jun 20, 2013 at 10:48 AM, Erick Erickson > <[email protected]> wrote: > > 1> MB, you're right. As usual, fresh eyes see stuff I _should_ have > > seen. If you look at the dump button, it's supposed to show how the > > figure was arrived at. If it doesn't, let me know. This still means it > > shouldn't be misleading.... > > > > 2> Right, this is the average number of bytes for the field you're > > defining per document. This won't particularly change the memory > > requirements _unless_ you have your document cache turned waaaay up. > > The tooltip when you hover over the entry field should be expanded to > > clarify this. > > > > Thanks for the feedback, I'll take care of this this weekend probably. > > > > Erick > > > > On Thu, Jun 20, 2013 at 12:38 AM, Dmitry Kan <[email protected]> > wrote: > >> Hello Erick, > >> > >> Tried your tool and have a couple of questions: > >> > >> 1. The total section: is the final figure presented in bytes (less > probable) > >> or megabytes (more probable)? > >> > >> -------------Total------------------------------------- > >> > >> TOTAL (MB): [3,895 (B)] > >> > >> 2. Defined a text field with custom type. Changing "Average text bytes > ONLY > >> if stored" didn't change the total. Probably correct? Does the field > mean > >> the raw byte size of a document? > >> > >> Thanks, > >> > >> Dmitry > >> > >> On 20 June 2013 09:52, Dmitry Kan <[email protected]> wrote: > >>> > >>> No worries. > >>> > >>> Otherwise the effort is very useful. This is the first question we > usually > >>> get from our superiors: how much RAM would we need to launch a feature? > >>> > >>> > >>> On 20 June 2013 00:36, Erick Erickson <[email protected]> wrote: > >>>> > >>>> Nope, never even noticed it until now. That's the right URL though, > >>>> typo and all.... > >>>> > >>>> Someday I may even fix it <G>... > >>>> > >>>> Thanks, > >>>> Erick > >>>> > >>>> On Wed, Jun 19, 2013 at 3:35 PM, Dmitry Kan <[email protected]> > >>>> wrote: > >>>> > Hi Erick, > >>>> > > >>>> > Is typo in the title on purpose? > >>>> > > >>>> > > >>>> > On 19 June 2013 15:09, Erick Erickson <[email protected]> > wrote: > >>>> >> > >>>> >> OK, I seem to have stalled on this. Over part of the winter, I put > >>>> >> together a Swing-based program to help estimate Solr/Lucene memory > >>>> >> requirements, with all the usual caveats see: > >>>> >> https://github.com/ErickErickson/SolrMemoryEsitmator. > >>>> >> > >>>> >> I have notes to myself that it's still deficient in several areas: > >>>> >> FieldValueCache estimates > >>>> >> tlog requirements > >>>> >> Memory required to re-open a searcher > >>>> >> Position and term vector memory requirements > >>>> >> And whatever I haven't thought about yet. > >>>> >> > >>>> >> Of course it builds on Grant's spreadsheet (reads "steals from it > >>>> >> shamelessly!") I'm hoping to have a friendlier interface. And _of > >>>> >> course_ I'd be willing to donate it to Solr as a > util/contrib/whatever > >>>> >> if it fits. > >>>> >> > >>>> >> So, what I'm about here is a few things: > >>>> >> > >>>> >> > Anyone who wants to try it feel free. The build instructions are > at > >>>> >> > the > >>>> >> > above, but the short form is to clone it, "ant jar" and "java > -jar > >>>> >> > dist/estimator.jar". Enter some field info and hit the "Add/Save" > >>>> >> > button > >>>> >> > then hit the "Dump calcs" button to see what it does currently. > >>>> >> > >>>> >> It also saves the estimates away in a file and shows all the steps > it > >>>> >> goes through to perform the calculations. It'll also make > rudimentary > >>>> >> field definitions from the entered data. You can come back to it > later > >>>> >> and add to what you've already done. > >>>> >> > >>>> >> > Make any improvements you see fit, particular to flesh out the > >>>> >> > deficiencies listed above. > >>>> >> > >>>> >> > Anyone who has, you know, graphic design/Swing skills please feel > >>>> >> > free > >>>> >> > to make it better. I'm a newbie as far as using Swing is > concerned, > >>>> >> > and the > >>>> >> > way I align buttons and checkboxes is pretty hacky. But it > works.... > >>>> >> > >>>> >> > Any suggestions anyone wants to make. Suggestions in code are > nicest > >>>> >> > of > >>>> >> > course, but algorithms for calculating, say, position and tv > memory > >>>> >> > usage > >>>> >> > would be great as well! Isolated code snippets that I could > >>>> >> > incorporate > >>>> >> > would be great too. > >>>> >> > >>>> >> > Any info where I've gotten the calculations wrong or don't show > >>>> >> > enough > >>>> >> > info to actually figure out whether they're correct or not. > >>>> >> > >>>> >> Note that the goal for this is to give a rough idea of memory > >>>> >> requirements and be easy to use. The spreadsheet is a bit daunting > to > >>>> >> someone who knows nothing about Solr so this might be an easier > way to > >>>> >> get into it. > >>>> >> > >>>> >> Thanks, > >>>> >> Erick > >>>> >> > >>>> >> > --------------------------------------------------------------------- > >>>> >> To unsubscribe, e-mail: [email protected] > >>>> >> For additional commands, e-mail: [email protected] > >>>> >> > >>>> > > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
