+1000 looks very interesting! I think we should a nice visualization app for our conference presentations... We might consider refreshing Visual [5], but personally I would love to see something new. A NodeJS client might be also a nice way to go but tracing how the data flows through the cluster... this would be something nice...
Thanks Sebastian [5] https://github.com/infinispan/visual On Mon, Jun 6, 2016 at 3:39 PM, Galder Zamarreño <gal...@redhat.com> wrote: > Hey guys, > > Sorry for the delay. I somehow read the replies but forgot to reply :) > > I think the target of Infispector and Zipkin based tracing would be > different, although there is some common ground. I think both Infispector > and Zipkin would be helpful in helping diagnose cluster performance issues > and get a general overview of how messages pass from one node to the other. > In terms of difference Infispector seems more geared towards education/qe > whereas Zipkin is more targeted as something we can run at runtime in > production for our users. > > Although both projects have different targets I think we'll be able to > take advantage of each others work, e.g. custom JGroups protocols points, > byteman/bytebuddy rules for intercept scenarions...etc. > > Great work leading all the Infispector work Tomas, looking forward to > demos/videos :D > > Cheers, > -- > Galder Zamarreño > Infinispan, Red Hat > > > On 10 May 2016, at 15:53, Radim Vansa <rva...@redhat.com> wrote: > > > > To complement this, MFT is a tool that won't offer any sleek charts or > > visualisations. It's tricky to use and understand - it's intended for > > developers as a tool for problem analysis. But it gets more in depth > > than InfiSpector, linking the information from different nodes, JFR > > events and so forth. > > > > R. > > > > On 05/10/2016 09:41 AM, Tomas Sykora wrote: > >> Hello Galder, and all! > >> It’s nice to communicate again via infinispan-dev after a while :) > >> > >> TL;DR: I can see some intersections with zipkin.io initiative goals > but InfiSpector seems to be much more “easier to handle and contribute to > at this moment” -- that suits more our student-related use case. Let’s > continue with the discussion :) > >> > >> Firstly, a short introduction into the context. Red Hat is running > Research & Development laboratory here in Brno at 2 biggest local > universities: Masaryk University, Faculty of Informatics (FI MU) and Brno > University of Technology, Faculty of Information Technologies (FIT VUT). > >> The aim is to better and sooner reach out to students, get them > involved into interesting project, show them open-source, git, git > workflows and many other things (project specific). An year ago I got > excited about this idea and started to think whether I can deliver such a > project. And I did. > >> > >> Team faces one big challenge and this is a time constraint. Students > are working on _several_ projects during their studies to fulfill courses’ > requirements to pass the semester. It’s hard for them to find additional > time to be coding even something else. Team managed that but slowly, that’s > understandable though. Designing InfiSpector infrastructure took us some > time (Kafka, Druid, NodeJS) + evaluation of these technologies + proof of > concepts. > >> > >> All 5 team members are 2nd year students of bachelor studies at FIT VUT > Brno. > >> Marek Ciz (https://github.com/mciz), also my very good friend from my > home town :) His primary domain is Druid, Kafka and infrastructure. > >> Vratislav Hais (https://github.com/vratislavhais) Primary domain is > front-end. > >> Jan Fitz (https://github.com/janfitz) Primary domain is front-end and > graphic design (also designed our logo). > >> Tomas Veskrna -- starting > >> Patrik Cigas -- starting > >> > >> At this moment we are very close to getting real data to be monitored > via web UI. It’s a matter of 1-2 months considering there is an examination > period happening now at the University. > >> > >> ******************* > >> What is InfiSpector and what we want to achieve: > >> > >> * We missed graphical representation of Infinispan nodes communication > so we want > >> -- To be able to spot possible issues AT THE FIRST LOOK (e.g. wait, > this should be coordinator, how is that possible he sends/receives only 10 > % of all messages?) > >> -- To demonstrate nicely what’s happening inside of ISPN cluster for > newcomers (to see how Infinispan nodes talk to each other to better > understand concepts) > >> -- To be using nice communication diagrams that describes flows like > (130 messages from node1 to node5 -- click to see them in detail, filter > out in more detail) > >> * We aimed for NON-invasive solution > >> -- No changes in Infinispan internal code > >> -- Just add custom JGroups protocol, gather data and send them where > you want [0] > >> * Provide historical recording of JGroups communication > >> * Help to analyze communication recording from big data point of view > >> -- No need to manually go through gigabytes of text trace logs > >> > >> Simplified InfiSpector architecture: > >> > >> Infinispan Cluster (JGroups with our protocol) ---> Apache Kafka ---> > Druid Database (using Kafka Firehose to inject Kafka Topic) <---> NodeJS > server back-end <---> front-end (AngularJS) > >> > >> What is coming out from custom JGroup protocol is a short JSON document > [1] with a timestamp, sending and receiving node, length of a message and > the message itself. Other stuff can be added easily. > >> > >> We will be able to easily answer queries like: > >> How many messages were sent from node1 to node3 during “last” 60 > seconds? > >> What are these messages? > >> How many of them were PutKeyValueCommands? > >> Filter out Heart beats (or even ignore them completely), etc. > >> > >> We don’t have any video recording yet but we are very close to that > point. From UI perspective we will be using these 2 charts: [2], [3]. > >> > >> > >> Talking about Infinispan 9 plans -- [4] was reported a month ago by you > Galder and we are working on InfiSpector actively let’s say 5 months -- it > looks like I should have advertised InfiSpector more, sooner, but I was > waiting for at least first working demo to start with blogging and videos > :) It’s good that you’ve noticed and that we are having this conversation > right now. > >> > >> To be honest I find http://zipkin.io/ initiative to be quite similar. > However, InfiSpector seems to be much more “easier” and not targeting at > performance analysis directly. Just adding one protocol at protocol stack > and you are good to go. We were thinking about putting Kafka and Druid > somewhere into the cloud (later) so users don’t need to start all that big > infrastructure at their local machines. > >> > >> I am very open to anything that will help us as a community to achieve > our common goal -- to be able to graphically monitor Infinispan > communication. > >> Additionally I would be _personally_ looking for something that is > easily achievable and is suitable for students to quickly learn new things > and quickly make meaningful contributions. > >> > >> Thanks! > >> Tomas > >> > >> [0] Achieved by custom JGroups protocol -- JGROUPS_TO_KAFKA protocol > has been implemented. This can be added at the end of JGroups stack and > every single message that goes through that is sent to Apache Kafka. > >> [1] > >> { > >> "direction": "receiving/up", > >> "src": "tsykora-19569", > >> "dest": "tsykora-27916", > >> "length": 182, > >> "timestamp": 1460302055376, > >> "message": "SingleRpcCommand{cacheName='___defaultcache', > command=PutKeyValueCommand{key=f6d52117-8a27-475e-86a7-002a54324615, > value=tsykora-19569, flags=null, putIfAbsent=false, > valueMatcher=MATCH_ALWAYS, > metadata=EmbeddedExpirableMetadata{lifespan=60000, maxIdle=-1, > version=null}, successful=true}}" > >> } > >> [2] http://bl.ocks.org/NPashaP/9796212 > >> [3] http://bl.ocks.org/mbostock/1046712 > >> [4] https://issues.jboss.org/browse/ISPN-6346 > >> > >> > >> > >> > >> ----- Original Message ----- > >>> From: "Galder Zamarreño" <gal...@redhat.com> > >>> To: "infinispan -Dev List" <infinispan-dev@lists.jboss.org>, "Tomas > Sykora" <tsyk...@redhat.com> > >>> Sent: Monday, May 9, 2016 5:06:06 PM > >>> Subject: Infispector > >>> > >>> Hi all, > >>> > >>> I've just noticed [1], @Thomas, it appears this is your baby? Could you > >>> explain in more detail what you are trying to achieve with this? Do > you have > >>> a video to show what exactly it does? > >>> > >>> Also, who's [2]? Curious to know who's working on this stuff :) > >>> > >>> The reason I'm interested in finding out a bit more about [1] is > because we > >>> have several efforts in the distributed monitoring/tracing area and > want to > >>> make sure we're not duplicating same effort. > >>> > >>> 1. Radim's Message Flow Tracer [3]: This is a project to tool for > tracing > >>> messages and control flow in JGroups/Infinispan using Byteman. > >>> > >>> 2. Zipkin effort [4]: The idea behind is to have a way to have > Infinispan > >>> cluster-wide tracing that uses Zipkin to capture and visualize where > time is > >>> spent within Infinispan. This includes both JGroups and other > components > >>> that could be time consuming, e.g. persistence. This will be main task > for > >>> Infinispan 9. This effort will use a lot of interception points Radim > has > >>> developed in [3] to tie together messages related to a request/tx > around the > >>> cluster. > >>> > >>> Does your effort fall within any of these spaces? > >>> > >>> Cheers, > >>> > >>> [1] https://github.com/infinispan/infispector > >>> [2] https://github.com/mciz > >>> [3] https://github.com/rvansa/message-flow-tracer > >>> [4] https://issues.jboss.org/browse/ISPN-6346 > >>> -- > >>> Galder Zamarreño > >>> Infinispan, Red Hat > >>> > >>> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev@lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > -- > > Radim Vansa <rva...@redhat.com> > > JBoss Performance Team > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev >
_______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
