Here is my 2 paisa on this discussion. > But we haven't spoken much about audience C, which will end up the biggest. > And that will include most of the developers at Netflix (I think only a few > of us are going to ssh onto instances and run the bcc tools, even though we > install them by default now).
Probably, audience C may further be divided in C1 (trace aggregation: live view/monitoring) and C2 (trace-collection: post-mortem analysis). C2 may be less than C1 though. Both are important IMO. I'd say there are multiple ways to insert BPF/bcc in currently available systems that allow visualizations with dashboards and GUI. With BPF tools, we have bare-bones of a powerful tracing framework already with us. We have all the probes we need. We need consumers/daemons to manage the incoming data. I believe, unless we have something visual that is presentable, it would be difficult to gather critical mass for eBPF adoption for audience C. > I did file this for Intel snap: > https://github.com/intelsdi-x/snap/issues/800 . Sounds like they may need > our help. We should also file something similar for collectd... What else? This may not be as uniform as collectd, but here are some projects that may benefit from eBPF based tracing events or have tried something similar in the past : Scope by Weaveworks: * HTTP Stats plugin : https://github.com/weaveworks-plugins/scope-http-statistics * tcptracer-bpf : https://github.com/weaveworks/tcptracer-bpf [Kinvolk's effort] Vector * There was some discussion that died down : https://github.com/Netflix/vector/issues/74 Graylog InfluxDB - time series event collection Prometheus Fluentd Grafana - time series visualizations Sysdig LTTng - if they agree to support events collected from eBPF. It already allows perf contexts (http://lttng.org/docs/v2.9/#doc-adding-context) Trace Compass (visualize stored CTF/XML/JSON or "whatever format" traces. We can define our own views in XML that we may package with eBPF tools - stuff which I am working on in my free time) HoneyComb Catapult (Chrome Trace Viewer) Another approach is that we decide on providing an interface that all projects can hook on to to get data. As an example, LTTng has a sessiond which manages the incoming data from tracing sessions, they have a relayd to relay the data to any consumer. Have a look : http://lttng.org/docs/v2.9/#doc-plumbing I may be biased, but this is a pretty sleek and robust architecture for any trace framework out there. In bbcc/BPF, we already have most of the trace recording things figured out with static and dynamic tracing support in kernel as well as userspace. The trace management part is missing. So we either tell others to develop their own, or provide a uniform interface. -- Suchakra _______________________________________________ iovisor-dev mailing list [email protected] https://lists.iovisor.org/mailman/listinfo/iovisor-dev
