Interesting! Looking forward to demos/videos... On 10/05/16 15:41, Tomas Sykora wrote: > Hello Galder, and all! > It’s nice to communicate again via infinispan-dev after a while :) > > TL;DR: I can see some intersections with zipkin.io initiative goals but > InfiSpector seems to be much more “easier to handle and contribute to at this > moment” -- that suits more our student-related use case. Let’s continue with > the discussion :) > > Firstly, a short introduction into the context. Red Hat is running Research & > Development laboratory here in Brno at 2 biggest local universities: Masaryk > University, Faculty of Informatics (FI MU) and Brno University of Technology, > Faculty of Information Technologies (FIT VUT). > The aim is to better and sooner reach out to students, get them involved into > interesting project, show them open-source, git, git workflows and many other > things (project specific). An year ago I got excited about this idea and > started to think whether I can deliver such a project. And I did. > > Team faces one big challenge and this is a time constraint. Students are > working on _several_ projects during their studies to fulfill courses’ > requirements to pass the semester. It’s hard for them to find additional time > to be coding even something else. Team managed that but slowly, that’s > understandable though. Designing InfiSpector infrastructure took us some time > (Kafka, Druid, NodeJS) + evaluation of these technologies + proof of concepts. > > All 5 team members are 2nd year students of bachelor studies at FIT VUT Brno. > Marek Ciz (https://github.com/mciz), also my very good friend from my home > town :) His primary domain is Druid, Kafka and infrastructure. > Vratislav Hais (https://github.com/vratislavhais) Primary domain is front-end. > Jan Fitz (https://github.com/janfitz) Primary domain is front-end and graphic > design (also designed our logo). > Tomas Veskrna -- starting > Patrik Cigas -- starting > > At this moment we are very close to getting real data to be monitored via web > UI. It’s a matter of 1-2 months considering there is an examination period > happening now at the University. > > ******************* > What is InfiSpector and what we want to achieve: > > * We missed graphical representation of Infinispan nodes communication so we > want > -- To be able to spot possible issues AT THE FIRST LOOK (e.g. wait, this > should be coordinator, how is that possible he sends/receives only 10 % of > all messages?) > -- To demonstrate nicely what’s happening inside of ISPN cluster for > newcomers (to see how Infinispan nodes talk to each other to better > understand concepts) > -- To be using nice communication diagrams that describes flows like (130 > messages from node1 to node5 -- click to see them in detail, filter out in > more detail) > * We aimed for NON-invasive solution > -- No changes in Infinispan internal code > -- Just add custom JGroups protocol, gather data and send them where you > want [0] > * Provide historical recording of JGroups communication > * Help to analyze communication recording from big data point of view > -- No need to manually go through gigabytes of text trace logs > > Simplified InfiSpector architecture: > > Infinispan Cluster (JGroups with our protocol) ---> Apache Kafka ---> Druid > Database (using Kafka Firehose to inject Kafka Topic) <---> NodeJS server > back-end <---> front-end (AngularJS) > > What is coming out from custom JGroup protocol is a short JSON document [1] > with a timestamp, sending and receiving node, length of a message and the > message itself. Other stuff can be added easily. > > We will be able to easily answer queries like: > How many messages were sent from node1 to node3 during “last” 60 seconds? > What are these messages? > How many of them were PutKeyValueCommands? > Filter out Heart beats (or even ignore them completely), etc. > > We don’t have any video recording yet but we are very close to that point. > From UI perspective we will be using these 2 charts: [2], [3]. > > > Talking about Infinispan 9 plans -- [4] was reported a month ago by you > Galder and we are working on InfiSpector actively let’s say 5 months -- it > looks like I should have advertised InfiSpector more, sooner, but I was > waiting for at least first working demo to start with blogging and videos :) > It’s good that you’ve noticed and that we are having this conversation right > now. > > To be honest I find http://zipkin.io/ initiative to be quite similar. > However, InfiSpector seems to be much more “easier” and not targeting at > performance analysis directly. Just adding one protocol at protocol stack and > you are good to go. We were thinking about putting Kafka and Druid somewhere > into the cloud (later) so users don’t need to start all that big > infrastructure at their local machines. > > I am very open to anything that will help us as a community to achieve our > common goal -- to be able to graphically monitor Infinispan communication. > Additionally I would be _personally_ looking for something that is easily > achievable and is suitable for students to quickly learn new things and > quickly make meaningful contributions. > > Thanks! > Tomas > > [0] Achieved by custom JGroups protocol -- JGROUPS_TO_KAFKA protocol has been > implemented. This can be added at the end of JGroups stack and every single > message that goes through that is sent to Apache Kafka. > [1] > { > "direction": "receiving/up", > "src": "tsykora-19569", > "dest": "tsykora-27916", > "length": 182, > "timestamp": 1460302055376, > "message": "SingleRpcCommand{cacheName='___defaultcache', > command=PutKeyValueCommand{key=f6d52117-8a27-475e-86a7-002a54324615, > value=tsykora-19569, flags=null, putIfAbsent=false, > valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=60000, > maxIdle=-1, version=null}, successful=true}}" > } > [2] http://bl.ocks.org/NPashaP/9796212 > [3] http://bl.ocks.org/mbostock/1046712 > [4] https://issues.jboss.org/browse/ISPN-6346 > > > > > ----- Original Message ----- >> From: "Galder Zamarreño" <gal...@redhat.com> >> To: "infinispan -Dev List" <infinispan-dev@lists.jboss.org>, "Tomas Sykora" >> <tsyk...@redhat.com> >> Sent: Monday, May 9, 2016 5:06:06 PM >> Subject: Infispector >> >> Hi all, >> >> I've just noticed [1], @Thomas, it appears this is your baby? Could you >> explain in more detail what you are trying to achieve with this? Do you have >> a video to show what exactly it does? >> >> Also, who's [2]? Curious to know who's working on this stuff :) >> >> The reason I'm interested in finding out a bit more about [1] is because we >> have several efforts in the distributed monitoring/tracing area and want to >> make sure we're not duplicating same effort. >> >> 1. Radim's Message Flow Tracer [3]: This is a project to tool for tracing >> messages and control flow in JGroups/Infinispan using Byteman. >> >> 2. Zipkin effort [4]: The idea behind is to have a way to have Infinispan >> cluster-wide tracing that uses Zipkin to capture and visualize where time is >> spent within Infinispan. This includes both JGroups and other components >> that could be time consuming, e.g. persistence. This will be main task for >> Infinispan 9. This effort will use a lot of interception points Radim has >> developed in [3] to tie together messages related to a request/tx around the >> cluster. >> >> Does your effort fall within any of these spaces? >> >> Cheers, >> >> [1] https://github.com/infinispan/infispector >> [2] https://github.com/mciz >> [3] https://github.com/rvansa/message-flow-tracer >> [4] https://issues.jboss.org/browse/ISPN-6346 >> -- >> Galder Zamarreño >> Infinispan, Red Hat >> >> > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev >
-- Bela Ban, JGroups lead (http://www.jgroups.org) _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
