That sounds interesting. To clarify the requirement, we can have a process that exposes the same JMX mbeans as the the real server and monitor can plug in to this process.
Thanks, Supun.. On Mon, Apr 22, 2013 at 10:57 AM, Josh Elser <[email protected]> wrote: > That would be pretty sweet, actually. Potentially parallel to what you > want to do, Supun, but cool nonetheless. > > I could see a lot of benefit by having some process that could emulate the > output from a non-trivially-sized Accumulo cluster on a single box. > > > On 4/22/13 10:43 AM, Eric Newton wrote: > >> You could mock the stats collection. >> >> -Eric >> >> >> On Mon, Apr 22, 2013 at 10:41 AM, David Medinets >> <[email protected]>**wrote: >> >> The average developer probably can't access a large cluster with hundred >>> of >>> nodes. Is there a way to simulate this? >>> >>> >>> On Mon, Apr 22, 2013 at 9:05 AM, Eric Newton <[email protected]> >>> wrote: >>> >>> Another thing to consider is scale. On large clusters (many hundreds of >>>> nodes), more data is not helpful for visualization. Instead, summaries, >>>> averages and outliers are important. >>>> >>>> For example, if one node is consistently slow, it is better to know that >>>> than to see one graph with low numbers in a sea of graphs. >>>> >>>> If the monitor collects information using JMX, collection time for each >>>> node would be a good thing to know, too. >>>> >>>> -Eric >>>> >>>> >>>> On Sun, Apr 21, 2013 at 10:00 PM, Josh Elser <[email protected]> >>>> >>> wrote: >>> >>>> Supun, >>>>> >>>>> Yup, very much so. Having a way to consume any and all metrics via JMX >>>>> would simplify things for any consumers (internal or external). >>>>> >>>>> >>>>> >>>>> On 04/21/2013 02:15 PM, Supun Kamburugamuva wrote: >>>>> >>>>> Hi Josh, >>>>>> >>>>>> Thanks for the suggestions. I'll incorporate these to the proposal. >>>>>> >>>>>> Another area I would like to work is on JMX. There is a Jira that says >>>>>> >>>>> to >>>> >>>>> replace the Monitor calls from Thrift to JMX (Accumulo 694). Do you >>>>>> >>>>> think >>>> >>>>> this is a good addition to the Monitor? >>>>>> >>>>>> Thanks, >>>>>> Supun.. >>>>>> >>>>>> >>>>>> On Sun, Apr 21, 2013 at 1:45 PM, Josh Elser <[email protected]> >>>>>> >>>>> wrote: >>>> >>>>> Supun, >>>>>> >>>>>>> Looks good! Can I make some suggestions/comments? >>>>>>> >>>>>>> For: "Per table plots: ACCUMULO-594", I'd also like to see minor >>>>>>> compactions, major compactions, index cache hit rate, and data cache >>>>>>> >>>>>> hit >>>> >>>>> rate per table (same graphs that are displayed system-wide when you >>>>>>> >>>>>> visit >>>> >>>>> http://${MONITOR_HOST}:50095/. >>>>>>> >>>>>>> For "Per tablet [server] plots", it would be neat if you could also >>>>>>> extract some general statistics like top N least performing, top N >>>>>>> highest >>>>>>> performing, etc. tablet servers. Ideally, this could correlate with >>>>>>> servers >>>>>>> that may be having problems :). >>>>>>> >>>>>>> Do you see these proposed changes as being sufficient for 3-4 months >>>>>>> >>>>>> of >>> >>>> 40hrs/week work? If you plan to really dig into these changes >>>>>>> >>>>>> (perhaps >>> >>>> reworking components of the monitor itself), I could perhaps see >>>>>>> >>>>>> this. >>> >>>> Do >>>> >>>>> you have any ideas for more lofty goals that you could pursue as >>>>>>> >>>>>> well? >>> >>>> I >>>> >>>>> don't want you/us to get one month into things and see you complete >>>>>>> everything we initially planned to accomplish :) >>>>>>> >>>>>>> - Josh >>>>>>> >>>>>>> >>>>>>> On 04/21/2013 10:37 AM, Supun Kamburugamuva wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>>> I would like to start writing the proposal for the GSoc. I've put >>>>>>>> together >>>>>>>> some initial high level goals of the project. Please let me know >>>>>>>> >>>>>>> what >>> >>>> I >>>> >>>>> can >>>>>>>> improve. >>>>>>>> >>>>>>>> Per table plots: Accumulo 594 >>>>>>>> --------------------- >>>>>>>> >>>>>>>> The goal of this is to display plots that explains the various >>>>>>>> activtities >>>>>>>> that happens per table. When we go to the tables page of the monitor >>>>>>>> >>>>>>> and >>>> >>>>> go >>>>>>>> to a specific table it displays some information in a table format. >>>>>>>> >>>>>>> We >>> >>>> can >>>>>>>> argument this information by showing graphs for >>>>>>>> >>>>>>>> 1. Ingest entries >>>>>>>> 2. Ingest data size >>>>>>>> 3. Scan entries >>>>>>>> 4. Scan data size >>>>>>>> >>>>>>>> Per tablet plots >>>>>>>> ---------------------- >>>>>>>> >>>>>>>> Same as in the table plots we can display information regarding >>>>>>>> >>>>>>> tablet >>> >>>> servers in the tablet server page. The plots will display the same >>>>>>>> information as table plots considering data per tablet server. >>>>>>>> >>>>>>>> Trace Visualization: Accumulo 1198 >>>>>>>> ---------------------------- >>>>>>>> >>>>>>>> Since we are displaying graphs about each tablet and each table we >>>>>>>> >>>>>>> can >>> >>>> add >>>>>>>> major and minor compaction graph to each table and each tablet. >>>>>>>> >>>>>>>> Or other option is to display this in a single graph in overview >>>>>>>> >>>>>>> page >>> >>>> with >>>>>>>> different graph lines for different tables and tablets. >>>>>>>> >>>>>>>> Server type information : Accumulo 807 >>>>>>>> ------------------------------******--- >>>>>>>> >>>>>>>> For displaying this informations we can add a new page and display >>>>>>>> >>>>>>> the >>> >>>> information as a table. The table should specify the network address >>>>>>>> >>>>>>> of >>>> >>>>> the >>>>>>>> server, server type, weather it is active or in-active etc. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Supun... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> > -- Supun Kamburugamuva Member, Apache Software Foundation; http://www.apache.org E-mail: [email protected]; Mobile: +1 812 369 6762 Blog: http://supunk.blogspot.com
