I would have some concerns with reducing the availability of stats or devaluing the tooling for analysis. Although exposing more stats via JMX is a great idea, most of the third party tooling I have seen is best for monitoring and not post analysis. Not investing in jVSD seems like it would make this post crash/issue analysis much more difficult in reviewing the complex interactions among multiple distributed nodes. For example correlating performance issues among 10 JVM's ( ie a GC in one node causing performance hiccups detected in another node) and be extremely difficult without using a tool that can consume and allow the overlay of multiple graphs of metrics we are collecting from many sources. The power here is in the ability to correlate multiple events from multiple JVM's into a single graphical view for debugging purposes and without this capability it will be significantly more difficult to understand the complex distributed behavior of Geode.
Currently custom stats are basically undocumented and difficult to use, removing them from the public API will probably have little impact on most users. Most users that want to do some monitoring can use JMX for their compentents but it would be helpful to have a method to add those values into the same stream as other stats/metrics for post issue analysis. *Vince Ford* GemFire Toolsmith Engineering Beaverton, OR USA http://www.pivotal.io Open Source Project Geode http://geode.apache.org/ <https://network.pivotal.io/products/project-geode> On Mon, Nov 20, 2017 at 11:46 PM, Jacob Barrett <jbarr...@pivotal.io> wrote: > Couldn’t agree more for the Java side of things. The first step is > deprecating the API for adding custom stats. > > > On Nov 20, 2017, at 11:13 PM, Swapnil Bawaskar <sbawas...@pivotal.io> > wrote: > > > > A lot of statistics we have are exposed over JMX. I think we should make > an > > effort to expose all the stats over JMX, making them consumable with > > existing tooling rather than investing in jVSD. > > > >> On Mon, Nov 20, 2017 at 2:32 PM Addison Huddy <ahu...@pivotal.io> > wrote: > >> > >> Thanks for the clarification Jake. So yes, I'm in agreement that we > >> should simplify the C++ API and remove the stats API from the C++ > client. > >> > >> \ah > >> > >> On Mon, Nov 20, 2017 at 10:23 AM, Jacob Barrett <jbarr...@pivotal.io> > >> wrote: > >> > >>> To clarify, the goal here is to remove access from the public API to > >> inject > >>> custom stats into our stats stream. We would still collect stats for > the > >>> internals of the client. > >>> > >>> The rational is multifaceted: > >>> > >>> 1) The C++ API is would need a non-trivial amount of time to modernize. > >> The > >>> current API uses pointers of pointers for maintaining collections. > There > >> is > >>> a strange cross relationship between components in the stats classes > >> which > >>> create unique pointer ownership questions. Rather than solving those > now > >>> and further dragging out the modernization of the C++ API we would drop > >> it > >>> and evaluated adding it back in a modern way in the future. Though I > >>> suspect adding it back in the future will never happen for the reasons > >>> below. > >>> > >>> 2) The storage format for our stats in proprietary to Geode and lacks > >> wide > >>> market adoption. > >>> > >>> 3) There are no modern tools for analyzing the statistics generated. > >> Geode > >>> lacks a tool for viewing or analyzing the statistics. Unless work is > >>> prioritized on completing the jVSD application then users are forced to > >>> write custom applications to extract the contents of the stats files. > >>> > >>> I support the removal from the Java public API for reasons 2 and 3. > >> Unless > >>> we put a full effort into creating the ecosystem around the stats > format > >> to > >>> make it usable we should remove it from the public API. I strongly > >>> encourage that we remove it internally too but that is for another > >>> discussion. > >>> > >>> -Jake > >>> > >>> > >>>> On Mon, Nov 20, 2017 at 9:43 AM Michael Stolz <mst...@pivotal.io> > wrote: > >>>> > >>>> I'm not clear on why we are removing stats gathering capability. > >>>> Do we know that customers aren't using this? > >>>> Is it badly broken? > >>>> > >>>> What is actually driving this work? > >>>> > >>>> -- > >>>> Mike Stolz > >>>> Principal Engineer, GemFire Product Lead > >>>> Mobile: +1-631-835-4771 <(631)%20835-4771> <(631)%20835-4771> > >>>> > >>>> On Mon, Nov 20, 2017 at 11:42 AM, Bruce Schuchardt < > >>> bschucha...@pivotal.io > >>>>> > >>>> wrote: > >>>> > >>>>> Should this be done for the Java caches as well? > >>>>> > >>>>> > >>>>>> On 11/17/17 11:48 AM, David Kimura wrote: > >>>>>> > >>>>>> I agree, a statistics interface seems beyond the scope of Geode > >> Native > >>>>>> client responsibility. Hiding or removing seems appropriate to me. > >>>>>> > >>>>>> Thanks, > >>>>>> David > >>>>>> > >>>>>> On Fri, Nov 17, 2017 at 11:29 AM, Ernest Burghardt > >>>>>> <eburgha...@pivotal.io> wrote: > >>>>>> > >>>>>>> +1 for removal > >>>>>>> > >>>>>>> On Thu, Nov 16, 2017 at 1:46 PM, Jacob Barrett < > >> jbarr...@pivotal.io> > >>>>>>> wrote: > >>>>>>> > >>>>>>> I want to open a discussion regarding the removal of > >>> StatisticsFactory > >>>>>>>> and > >>>>>>>> related APIs from the public API. I can't see that we would want > >> the > >>>>>>>> Geode > >>>>>>>> Native client to be a first class statistics/metrics gathering > >> API. > >>>>>>>> There > >>>>>>>> are plenty of other first class players in this space. If this > >>> isn't a > >>>>>>>> feature of the client then I suggest it be moved internally. It’s > >>>> highly > >>>>>>>> unlikely it’s being used but in the case that it is we can > >> consider > >>>>>>>> moving > >>>>>>>> it back after some serious refactoring as it relies on an over > >>>>>>>> abundance of > >>>>>>>> raw pointers. Rather than spend time refactoring it now let’s just > >>>> hide > >>>>>>>> it > >>>>>>>> away. > >>>>>>>> > >>>>>>>> -Jake > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>> > >>>> > >>> > >> >
