Thanks ll for the info here. I am going to press forward with investigating HTrace for the Nutch 2.X Search Stack. Lewis
On Fri, Feb 27, 2015 at 11:58 AM, Colin P. McCabe <[email protected]> wrote: > Hi Lewis, > > Good questions. I would say HTrace differs from TRACE logging (or > other single-node metrics, JMX, audit logs, etc.) in that it pulls > together information from across the cluster. This is something that > is a major pain point when using a distributed system such as HDFS. > Just to diagnose a slow write, you might have to match up logs from a > client log and the logs of 3 different datanodes. The big idea behind > htrace is two things: integrating those logging sources, and using > sampling to instrument performance in production. The main thing > htrace deals with is "spans" which are lengths of time. > > We're working on a web UI that will allow people to search for spans > by time, duration, and name (among other things). It's not quite > finished now (hoping to have something usable in HTrace 3.2.0 or maybe > 3.3.0... but abe can comment more on that.) > > Here's an early screenshot (probably way out of date now): > > https://issues.apache.org/jira/secure/attachment/12689757/Search%20page%20skeleton%20-%200.png > > There is also a plan to create a visualization of parent/child > relationships on the web UI, by using the d3 library (which can draw > graphs, and do many other things besides.) > > In the meantime, there's an option to product a graphviz file from a > file containing span JSON. That way you can draw a graph of > parent/child relationships with the "dot" tool, available on Linux. > Uh... unfortunately it's broken right now... let me file a JIRA for > that :P This is a very new feature, got added earlier this week. > > The web UI is a great place to get involved right now... there is a > lot of work going on there and we've been adding new contributors. > > Colin > > On Thu, Feb 26, 2015 at 1:46 PM, Lewis John Mcgibbney > <[email protected]> wrote: > > Hi Nick, > > > > Grand. Thank you > > > > What is visualization looking like right now? It there currently a > > mechanism for visualizing HTrace structures? > > Is it worth considering posting something like this as a GSoC project is > > one does not currently exist? > > Thanks > > Lewis > > > > > > On Thu, Feb 26, 2015 at 1:31 PM, Nick Dimiduk <[email protected]> > wrote: > > > >> Hi Lewis, > >> > >> > -- *Lewis*
