Thanks for the suggestion, David. I'm not familiar with Yammer's metrics libraries. I would like to keep this project limited to Ambari and Accumulo, but I will look at the links you included and learn more about that library, for future consideration.
Thanks again, Andres On Sat, Apr 13, 2013 at 1:44 PM, David Schlosnagle <[email protected]>wrote: > In regards to #6, I'm curious if you've considered using Yammer's Apache > licensed metrics library [1] for capturing stats as it offers many > different options for consuming and monitoring produced metrics. Coda Hale > gives a good overview in this talk [2]. > > [1]: http://metrics.codahale.com/ > [2]: http://pivotallabs.com/139-metrics-metrics-everywhere/ > > - Dave > > On Apr 13, 2013, at 12:15 PM, Andres Danter <[email protected]> wrote: > > > Awesome suggestions! Thank you, Josh. > > On Apr 12, 2013 10:01 PM, "Josh Elser" <[email protected]> wrote: > > > >> Andres, > >> > >> Looks good. Glad to see a lot of good ideas here already. I have a > couple > >> of refinements if you don't mind. > >> > >> 3. Have the ability to schedule /graceful/ restart of Accumulo... > >> > >> It would be really cool to have the option of bringing down an entire > >> cluster or do a rolling-restart (iteratively bring down a node, let > things > >> quiesce, then restart that node) to try to ensure 100% availability. > >> Accumulo handles most of this for you -- it's more a matter of waiting > for > >> any unhosted tablets to be re-assigned after a tabletserver is stopped. > >> > >> 4. Push Accumulo code ... > >> > >> With 1.5 releases, there's a VFS classloader built in which allows for > >> various types of file systems to be used to load files onto the > classpath. > >> The most interesting of which probably include HDFS, HTTP(S), and > (S)FTP. > >> It would be neat to support not only loading classes from the local > >> filesystem but also by a variety of other means. > >> > >> 6. Gather system and/or application metrics.. > >> > >> Just be aware that this is a very open-ended subject and, most likely, > >> could be a full GSOC task on its own. You probably want to think about > what > >> might be best for Accumulo (starting with what the monitor and master > >> already supply) and how best you can integrate with Ambari. > >> > >> Again, overall, sounds awesome. I look forward to working with you. > >> > >> > >> On 04/12/2013 05:58 PM, Andres Danter wrote: > >> > >>> Thanks, Billie. I'm liking this task more and more. I would love to > take > >>> this on for GSoC. Please run it by the Ambari PMC. I will start > drafting > >>> the proposal this weekend. > >>> > >>> Just to be clear, the goal would be to integrate Ambari with Accumulo > in > >>> order to achieve the following capabilities (I've expanded David's > >>> original > >>> list of features): > >>> > >>> 1. Control (start/stop) Accumulo processes on any cluster node > >>> 2. Have the ability to edit Accumulo configuration files via a > web-based > >>> editor (I don't think Ambari provides this. Please let me know if I'm > >>> wrong) > >>> 3. Have the ability to schedule restart of Accumulo processes after a > >>> configuration change > >>> 4. Push Accumulo code and configuration files to a new cluster node > >>> 5. Gracefully remove a node from a cluster > >>> 6. Gather system and/or application metrics that can be used to > monitor > >>> the system and perhaps create reports on usability > >>> 7. Be able to monitor and control the Accumulo cluster from a > web-based > >>> GUI (this kind of goes with #2 above) > >>> 8. Allow read-only monitor sessions, as well as full-admin sessions > and > >>> have the ability to strictly manage who can access either. (this I got > >>> from ACCUMULO-196) > >>> 9. Provide the ability to authenticate user sessions with SSL > >>> (see ACCUMULO-196) > >>> > >>> It seems like the product of this work would largely replace the > current > >>> Accumulo Monitor. If that is the case, are there features of the > monitor > >>> which you would like to see integrated? > >>> > >>> I will have about 3 months to complete this work (actually about > 220-240 > >>> hours), so that list is doable. Most of the work will involve the GUI > >>> portions, unless Ambari already comes with much of the functionality. > If > >>> the list is too short, please let me know. > >>> > >>> I guess I should sign up to the Ambari mailing list as well, right? > >>> > >>> Thanks again and I look forward to being your mentee and to working > with > >>> the Accumulo community. > >>> > >>> Andres > >>> > >>> > >>> On Fri, Apr 12, 2013 at 3:43 PM, David Medinets < > [email protected] > >>>> **wrote: > >>> > >>> The project is open source ... we can't stop you! > >>>> > >>>> > >>>> On Fri, Apr 12, 2013 at 2:25 PM, Andres Danter <[email protected]> > >>>> wrote: > >>>> > >>>> Could the work of integrating Accumulo with Ambari be something that > you > >>>>> would allow a student to tackle for GSoC? I don't want to step on > >>>> anyone's > >>>> > >>>>> planned work, but such a task really interests me. I want to tackle > >>>>> something with some meat, if you know what I mean. > >>>>> > >>>>> > >>>>> > >>>>> On Fri, Apr 12, 2013 at 11:47 AM, David Medinets > >>>>> <[email protected]>**wrote: > >>>>> > >>>>> Thanks for the reminder. I had forgotten the name of Ambari. > >>>>>> > >>>>>> > >>>>>> On Fri, Apr 12, 2013 at 10:37 AM, Keith Turner <[email protected]> > >>>>> wrote: > >>>> > >>>>> On Fri, Apr 12, 2013 at 10:29 AM, David Medinets > >>>>>>> <[email protected]> wrote: > >>>>>>> > >>>>>>>> It might be interesting to have a tool which could: > >>>>>>>> > >>>>>>>> a) shut down any accumulo process on any server. > >>>>>>>> b) start any accumulo process on any server. > >>>>>>>> c) edit the various configuration files through a web interface. > >>>>>>>> d) automate restart accumulo processes, as needed, after > >>>>>>> configuration > >>>>> > >>>>>> change. > >>>>>>>> e) push accumulo code and configuration to a new server. > >>>>>>>> f) gracefully shutdown and remove a server from the cluster. > >>>>>>>> > >>>>>>>> Anything else? If this sounds like a good idea, let's create a > JIRA > >>>>>>> ticket > >>>>>>> > >>>>>>>> with the right tags. > >>>>>>> This sounds like Apache Ambari, which we should possibly integrate > >>>>>>> with. See ACCUMULO-136 > >> >
