Re: [Gluster-devel] New project on the Forge - gstatus
, 2014 5:06:27 AM Subject: Re: [Gluster-devel] New project on the Forge - gstatus Hi Paul, I'm really not sure, if this can be done in python(at least comfortably). Maybe we can tread on the same path as Justin's glusterflow in python. But I don't think, all the io-stats counters will be available with the way how Justin's used Jeff Darcy's previous work to build his tool. I can be wrong. My knowledge is a bit incomplete and based on very less experience as a user and an amateur Gluster developer. Please do correct me, if I can be. Regards Vipul Nayyar ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] New project on the Forge - gstatus
-de...@nongnu.org Sent: Thursday, 20 February, 2014 5:06:27 AM Subject: Re: [Gluster-devel] New project on the Forge - gstatus Hi Paul, I'm really not sure, if this can be done in python(at least comfortably). Maybe we can tread on the same path as Justin's glusterflow in python. But I don't think, all the io-stats counters will be available with the way how Justin's used Jeff Darcy's previous work to build his tool. I can be wrong. My knowledge is a bit incomplete and based on very less experience as a user and an amateur Gluster developer. Please do correct me, if I can be. Regards Vipul Nayyar ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel -- Sent from my Android device with K-9 Mail. Please excuse my brevity.___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] New project on the Forge - gstatus
Avati that's awesome! I didn't know that the meta xlator could do that already. On Sat, May 17, 2014 at 9:30 AM, Anand Avati av...@gluster.org wrote: KP, Vipul, It will be awesome to get io-stats like instrumentation on the client side. Here are some further thoughts on how to implement that. If you have a recent git HEAD build, I would suggest that you explore the latency stats on the client side exposed through meta at $MNT/.meta/graphs/active/$xlator/profile. You can enable latency measurement with echo 1 $MNT/.meta/measure_latency. I would suggest extending these stats with the extra ones io-stats has, and make glusterfsiostats expose these stats. If you can compare libglusterfs/src/latency.c:gf_latency_begin(), gf_latency_end() and gf_latency_udpate() with the macros in io-stats.c UPDATE_PROFILE_STATS() and START_FOP_LATENCY(), you will quickly realize how a lot of logic is duplicated between io-stats and latency.c. If you can enhance latency.c and make it capture the remaining stats what io-stats is capturing, the benefits of this approach would be: - stats are already getting captured at all xlator levels, and not just at the position where io-stats is inserted - file like interface makes the stats more easily inspectable and consumable, and updated on the fly - conforms with the way rest of the internals are exposed through $MNT/.meta In order to this, you might want to look into: - latency.c as of today captures fop count, mean latency, total time, whereas io-stats measures these along with min-time, max-time and block-size histogram. - extend gf_proc_dump_latency_info() to dump the new stats - either prettify that output like 'volume profile info' output, or JSONify it like xlators/meta/src/frames-file.c - add support for cumulative vs interval stats (store an extra copy of this-latencies[]) etc.. Thanks! On Fri, Apr 25, 2014 at 9:09 PM, Krishnan Parthasarathi kpart...@redhat.com wrote: [Resending due to gluster-devel mailing list issue] Apologies for the late reply. glusterd uses its socket connection with brick processes (where io-stats xlator is loaded) to gather information from io-stats via an RPC request. This facility is restricted to brick processes as it stands today. Some background ... io-stats xlator is loaded, both in GlusterFS mounts and brick processes. So, we have the capabilities to monitor I/O statistics on both sides. To collect I/O statistics at the server side, we have # gluster volume profile VOLNAME [start | info | stop] AND #gluster volume top VOLNAME info [and other options] We don't have a usable way of gathering I/O statistics (not monitoring, though the counters could be enhanced) at the client-side, ie. for a given mount point. This is the gap glusterfsiostat aims to fill. We need to remember that the machines hosting GlusterFS mounts may not have glusterd installed on them. We are considering rrdtool as a possible statistics database because it seems like a natural choice for storing time-series data. rrdtool is capable of answering high-level statistical queries on statistics that were logged in it by io-stats xlator over and above printing running counters periodically. Hope this gives some more clarity on what we are thinking. thanks, Krish - Original Message - Probably me not understanding. the comment iostats making data available to glusterd over RPC - is what I latched on to. I wondered whether this meant that a socket could be opened that way to get at the iostats data flow. Cheers, PC - Original Message - From: Vipul Nayyar nayyar_vi...@yahoo.com To: Paul Cuzner pcuz...@redhat.com, Krishnan Parthasarathi kpart...@redhat.com Cc: Vijay Bellur vbel...@redhat.com, gluster-devel gluster-de...@nongnu.org Sent: Thursday, 20 February, 2014 5:06:27 AM Subject: Re: [Gluster-devel] New project on the Forge - gstatus Hi Paul, I'm really not sure, if this can be done in python(at least comfortably). Maybe we can tread on the same path as Justin's glusterflow in python. But I don't think, all the io-stats counters will be available with the way how Justin's used Jeff Darcy's previous work to build his tool. I can be wrong. My knowledge is a bit incomplete and based on very less experience as a user and an amateur Gluster developer. Please do correct me, if I can be. Regards Vipul Nayyar ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo
Re: [Gluster-devel] New project on the Forge - gstatus
KP, Vipul, It will be awesome to get io-stats like instrumentation on the client side. Here are some further thoughts on how to implement that. If you have a recent git HEAD build, I would suggest that you explore the latency stats on the client side exposed through meta at $MNT/.meta/graphs/active/$xlator/profile. You can enable latency measurement with echo 1 $MNT/.meta/measure_latency. I would suggest extending these stats with the extra ones io-stats has, and make glusterfsiostats expose these stats. If you can compare libglusterfs/src/latency.c:gf_latency_begin(), gf_latency_end() and gf_latency_udpate() with the macros in io-stats.c UPDATE_PROFILE_STATS() and START_FOP_LATENCY(), you will quickly realize how a lot of logic is duplicated between io-stats and latency.c. If you can enhance latency.c and make it capture the remaining stats what io-stats is capturing, the benefits of this approach would be: - stats are already getting captured at all xlator levels, and not just at the position where io-stats is inserted - file like interface makes the stats more easily inspectable and consumable, and updated on the fly - conforms with the way rest of the internals are exposed through $MNT/.meta In order to this, you might want to look into: - latency.c as of today captures fop count, mean latency, total time, whereas io-stats measures these along with min-time, max-time and block-size histogram. - extend gf_proc_dump_latency_info() to dump the new stats - either prettify that output like 'volume profile info' output, or JSONify it like xlators/meta/src/frames-file.c - add support for cumulative vs interval stats (store an extra copy of this-latencies[]) etc.. Thanks! On Fri, Apr 25, 2014 at 9:09 PM, Krishnan Parthasarathi kpart...@redhat.com wrote: [Resending due to gluster-devel mailing list issue] Apologies for the late reply. glusterd uses its socket connection with brick processes (where io-stats xlator is loaded) to gather information from io-stats via an RPC request. This facility is restricted to brick processes as it stands today. Some background ... io-stats xlator is loaded, both in GlusterFS mounts and brick processes. So, we have the capabilities to monitor I/O statistics on both sides. To collect I/O statistics at the server side, we have # gluster volume profile VOLNAME [start | info | stop] AND #gluster volume top VOLNAME info [and other options] We don't have a usable way of gathering I/O statistics (not monitoring, though the counters could be enhanced) at the client-side, ie. for a given mount point. This is the gap glusterfsiostat aims to fill. We need to remember that the machines hosting GlusterFS mounts may not have glusterd installed on them. We are considering rrdtool as a possible statistics database because it seems like a natural choice for storing time-series data. rrdtool is capable of answering high-level statistical queries on statistics that were logged in it by io-stats xlator over and above printing running counters periodically. Hope this gives some more clarity on what we are thinking. thanks, Krish - Original Message - Probably me not understanding. the comment iostats making data available to glusterd over RPC - is what I latched on to. I wondered whether this meant that a socket could be opened that way to get at the iostats data flow. Cheers, PC - Original Message - From: Vipul Nayyar nayyar_vi...@yahoo.com To: Paul Cuzner pcuz...@redhat.com, Krishnan Parthasarathi kpart...@redhat.com Cc: Vijay Bellur vbel...@redhat.com, gluster-devel gluster-de...@nongnu.org Sent: Thursday, 20 February, 2014 5:06:27 AM Subject: Re: [Gluster-devel] New project on the Forge - gstatus Hi Paul, I'm really not sure, if this can be done in python(at least comfortably). Maybe we can tread on the same path as Justin's glusterflow in python. But I don't think, all the io-stats counters will be available with the way how Justin's used Jeff Darcy's previous work to build his tool. I can be wrong. My knowledge is a bit incomplete and based on very less experience as a user and an amateur Gluster developer. Please do correct me, if I can be. Regards Vipul Nayyar ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
