Gaaah, I've been looking at this more today and it's completely not ready for prime-time. Numbers are bogus, so don't rely on it much.
On Thu, Jun 20, 2013 at 10:48 AM, Erick Erickson <[email protected]> wrote: > 1> MB, you're right. As usual, fresh eyes see stuff I _should_ have > seen. If you look at the dump button, it's supposed to show how the > figure was arrived at. If it doesn't, let me know. This still means it > shouldn't be misleading.... > > 2> Right, this is the average number of bytes for the field you're > defining per document. This won't particularly change the memory > requirements _unless_ you have your document cache turned waaaay up. > The tooltip when you hover over the entry field should be expanded to > clarify this. > > Thanks for the feedback, I'll take care of this this weekend probably. > > Erick > > On Thu, Jun 20, 2013 at 12:38 AM, Dmitry Kan <[email protected]> wrote: >> Hello Erick, >> >> Tried your tool and have a couple of questions: >> >> 1. The total section: is the final figure presented in bytes (less probable) >> or megabytes (more probable)? >> >> -------------Total------------------------------------- >> >> TOTAL (MB): [3,895 (B)] >> >> 2. Defined a text field with custom type. Changing "Average text bytes ONLY >> if stored" didn't change the total. Probably correct? Does the field mean >> the raw byte size of a document? >> >> Thanks, >> >> Dmitry >> >> On 20 June 2013 09:52, Dmitry Kan <[email protected]> wrote: >>> >>> No worries. >>> >>> Otherwise the effort is very useful. This is the first question we usually >>> get from our superiors: how much RAM would we need to launch a feature? >>> >>> >>> On 20 June 2013 00:36, Erick Erickson <[email protected]> wrote: >>>> >>>> Nope, never even noticed it until now. That's the right URL though, >>>> typo and all.... >>>> >>>> Someday I may even fix it <G>... >>>> >>>> Thanks, >>>> Erick >>>> >>>> On Wed, Jun 19, 2013 at 3:35 PM, Dmitry Kan <[email protected]> >>>> wrote: >>>> > Hi Erick, >>>> > >>>> > Is typo in the title on purpose? >>>> > >>>> > >>>> > On 19 June 2013 15:09, Erick Erickson <[email protected]> wrote: >>>> >> >>>> >> OK, I seem to have stalled on this. Over part of the winter, I put >>>> >> together a Swing-based program to help estimate Solr/Lucene memory >>>> >> requirements, with all the usual caveats see: >>>> >> https://github.com/ErickErickson/SolrMemoryEsitmator. >>>> >> >>>> >> I have notes to myself that it's still deficient in several areas: >>>> >> FieldValueCache estimates >>>> >> tlog requirements >>>> >> Memory required to re-open a searcher >>>> >> Position and term vector memory requirements >>>> >> And whatever I haven't thought about yet. >>>> >> >>>> >> Of course it builds on Grant's spreadsheet (reads "steals from it >>>> >> shamelessly!") I'm hoping to have a friendlier interface. And _of >>>> >> course_ I'd be willing to donate it to Solr as a util/contrib/whatever >>>> >> if it fits. >>>> >> >>>> >> So, what I'm about here is a few things: >>>> >> >>>> >> > Anyone who wants to try it feel free. The build instructions are at >>>> >> > the >>>> >> > above, but the short form is to clone it, "ant jar" and "java -jar >>>> >> > dist/estimator.jar". Enter some field info and hit the "Add/Save" >>>> >> > button >>>> >> > then hit the "Dump calcs" button to see what it does currently. >>>> >> >>>> >> It also saves the estimates away in a file and shows all the steps it >>>> >> goes through to perform the calculations. It'll also make rudimentary >>>> >> field definitions from the entered data. You can come back to it later >>>> >> and add to what you've already done. >>>> >> >>>> >> > Make any improvements you see fit, particular to flesh out the >>>> >> > deficiencies listed above. >>>> >> >>>> >> > Anyone who has, you know, graphic design/Swing skills please feel >>>> >> > free >>>> >> > to make it better. I'm a newbie as far as using Swing is concerned, >>>> >> > and the >>>> >> > way I align buttons and checkboxes is pretty hacky. But it works.... >>>> >> >>>> >> > Any suggestions anyone wants to make. Suggestions in code are nicest >>>> >> > of >>>> >> > course, but algorithms for calculating, say, position and tv memory >>>> >> > usage >>>> >> > would be great as well! Isolated code snippets that I could >>>> >> > incorporate >>>> >> > would be great too. >>>> >> >>>> >> > Any info where I've gotten the calculations wrong or don't show >>>> >> > enough >>>> >> > info to actually figure out whether they're correct or not. >>>> >> >>>> >> Note that the goal for this is to give a rough idea of memory >>>> >> requirements and be easy to use. The spreadsheet is a bit daunting to >>>> >> someone who knows nothing about Solr so this might be an easier way to >>>> >> get into it. >>>> >> >>>> >> Thanks, >>>> >> Erick >>>> >> >>>> >> --------------------------------------------------------------------- >>>> >> To unsubscribe, e-mail: [email protected] >>>> >> For additional commands, e-mail: [email protected] >>>> >> >>>> > >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>> >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
