Hi Sri,
I did something similar a while back for a hackday project. My approach was
to expose all of GemFire JMX metrics with jmxtrans which can automatically
write to InfluxDB as a Graphite source. I like this approach because a
GemFire member can be instrumented without adding *any* additional code.
FWIW I'm attaching the jmxtrans.xml I used to expose a couple of standard
JMX JVM attributes.
--Jens
On Mon, Sep 26, 2016 at 3:10 PM, sri man <manv...@gmail.com> wrote:
> Hi,
>
> I am trying to experiment with exposing Geode stats to Grafana
> <http://grafana.org/> using a the time-series database InfluxDB
> <https://www.influxdata.com/time-series-platform/influxdb/>.
>
> Facing the below exception when I start the server after enabling stats and
> including influxDB java client <https://github.com/
> influxdata/influxdb-java> in
> the project.
> I am new to gradle (and also the above technologies ! ), not sure if I am
> doing something wrong here w.r.t including the influxdb-java client.
>
> When I enable statistics, the server doesn't start(the .... after the
> start server command doesn't seem to stop), and I see the below exception
> in the server logs
>
> [warning 2016/09/26 17:47:45.408 EDT geode-server1 <main> tid=0x1]
> Statistics sampler is not available
> *java.lang.IllegalStateException: Statistics sampler is not available*
> at
> org.apache.geode.internal.statistics.SampleCollector.
> getStatMonitorHandler(SampleCollector.java:127)
> at
> org.apache.geode.internal.statistics.StatisticsMonitor.
> getStatMonitorHandler(StatisticsMonitor.java:139)
> at
> org.apache.geode.internal.statistics.StatisticsMonitor.
> addListener(StatisticsMonitor.java:83)
> at
> org.apache.geode.management.internal.beans.stats.MBeanStatsMonitor.
> addStatisticsToMonitor(MBeanStatsMonitor.java:59)
> at
> org.apache.geode.management.internal.beans.MemberMBeanBridge.
> addCacheStats(MemberMBeanBridge.java:470)
> at
> org.apache.geode.management.internal.beans.MemberMBeanBridge.init(
> MemberMBeanBridge.java:420)
> at
> org.apache.geode.management.internal.beans.ManagementAdapter.
> handleCacheCreation(ManagementAdapter.java:161)
> at
> org.apache.geode.management.internal.beans.ManagementListener.handleEvent(
> ManagementListener.java:119)
> at
> org.apache.geode.distributed.internal.InternalDistributedSystem.
> notifyResourceEventListeners(InternalDistributedSystem.java:2077)
> at
> org.apache.geode.distributed.internal.InternalDistributedSystem.
> handleResourceEvent(InternalDistributedSystem.java:525)
> at
> org.apache.geode.internal.cache.GemFireCacheImpl.
> initialize(GemFireCacheImpl.java:1113)
> at
> org.apache.geode.internal.cache.GemFireCacheImpl.
> basicCreate(GemFireCacheImpl.java:765)
> at
> org.apache.geode.internal.cache.GemFireCacheImpl.create(
> GemFireCacheImpl.java:752)
> at org.apache.geode.cache.CacheFactory.create(
> CacheFactory.java:181)
> at org.apache.geode.cache.CacheFactory.create(
> CacheFactory.java:231)
> at
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProv
> ider.createCache(DefaultServerLauncherCacheProvider.java:55)
> at
> org.apache.geode.distributed.ServerLauncher.createCache(
> ServerLauncher.java:783)
> at
> org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:703)
> at
> org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:633)
> at
> org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:184)
> For now I have changed the method (private writeSample
> in StatArchiveWriter.java) to write to influxDB.
>
> *Geode Properties:*
> statistic-sampling-enabled=true
> statistic-archive-file=statsFile.gfs
>
> When I remove the above properties from the properties file the server
> starts up fine, which makes me think that my gradle changes should be fine.
>
> *Gradle changes:*
>
> 1. I have included the below line in geode-core/build.gradle
> <https://github.com/apache/incubator-geode/blob/develop/
> geode-core/build.gradle#L39>
> dependencies.
> compile 'org.influxdb:influxdb-java:' + project.'influxdb-client-
> version'
>
> 2. I have included the dependency version here
> https://github.com/apache/incubator-geode/blob/develop/
> gradle/dependency-versions.properties
>
> 3. Have added the influxdb jar name here
> https://github.com/apache/incubator-geode/blob/develop/
> geode-assembly/src/test/resources/expected_jars.txt
>
>
> Please let me know why one would see exception
> "java.lang.IllegalStateException: Statistics sampler is not available".
>
> Any feedback on the approach/usefulness of using InfluxDB-Grafana for geode
> stats is appreciated.
>
>
> Thanks,
> Srikanth Manvi
>
<jmxtrans-agent>
<queries>
<!-- OS -->
<query objectName="java.lang:type=OperatingSystem" attribute="SystemLoadAverage" resultAlias="os.systemLoadAverage"/>
<!-- JVM -->
<query objectName="java.lang:type=Memory" attribute="HeapMemoryUsage" key="used"
resultAlias="jvm.heapMemoryUsage.used"/>
<query objectName="java.lang:type=Memory" attribute="HeapMemoryUsage" key="committed"
resultAlias="jvm.heapMemoryUsage.committed"/>
<query objectName="java.lang:type=Memory" attribute="NonHeapMemoryUsage" key="used"
resultAlias="jvm.nonHeapMemoryUsage.used"/>
<query objectName="java.lang:type=Memory" attribute="NonHeapMemoryUsage" key="committed"
resultAlias="jvm.nonHeapMemoryUsage.committed"/>
<query objectName="java.lang:type=ClassLoading" attribute="LoadedClassCount" resultAlias="jvm.loadedClasses"/>
<query objectName="java.lang:type=Threading" attribute="ThreadCount" resultAlias="jvm.thread"/>
<query objectName="GemFire:service=System,type=Distributed" attribute="TotalHeapSize" resultAlias="totalHeapSize"/>
<query objectName="GemFire:service=System,type=Distributed" attribute="UsedHeapSize" resultAlias="usedHeapSize"/>
</queries>
<outputWriter class="org.jmxtrans.agent.GraphitePlainTextTcpOutputWriter">
<host>localhost</host>
<port>2003</port>
<namePrefix>gemfire.</namePrefix>
</outputWriter>
<outputWriter class="org.jmxtrans.agent.ConsoleOutputWriter"/>
<collectIntervalInSeconds>20</collectIntervalInSeconds>
</jmxtrans-agent>
