I am trying to setup Ganglia for Monitoring Accumulo 1.7.0 Cluster
having 5 nodes. I also configured ganglia for Accumulo 1.6.1 standalone
cluster which is working fine and shows all the required metrices
related to Accumulo.
After scanning jmxtrans.log, I found that the its not exexuting quring
properly. It starts the job and finish in next line
[27 Jan 2016 17:38:15] [ServerScheduler_Worker-1] 3960536 DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
Server [host=localhost, port=9006,
url=service:jmx:rmi:///jndi/rmi://localhost:9006/jmxrmi,
cronExpression=null, numQueryThreads=null]
[27 Jan 2016 17:38:15] [ServerScheduler_Worker-1] 3960542 DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
Server [host=localhost, port=9006,
url=service:jmx:rmi:///jndi/rmi://localhost:9006/jmxrmi,
cronExpression=null, numQueryThreads=null]
But this not the case with Accumulo 1.6.1 cluster it executes the query
and send the ganglia metrices which I can see on ganglia web ui...
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360736 DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
Server [host=mohit, port=9006,
url=service:jmx:rmi:///jndi/rmi://mohit:9006/jmxrmi,
cronExpression=null, numQueryThreads=null]
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360742 DEBUG
(com.googlecode.jmxtrans.util.JmxUtils:195) - Executing queryName:
accumulo.server.metrics:instance=tserver,name=TabletServerMBean,service=TServerInfo
from query: Query
[obj=accumulo.server.metrics:service=TServerInfo,name=TabletServerMBean,instance=tserver,
resultAlias=Accumulo, attr=[Entries, EntriesInMemory, Ingest,
MajorCompactions, MajorCompactionsQueued, MinorCompactions,
MinorCompactionsQueued, OnlineCount, OpeningCount, Queries,
UnopenedCount, TotalMinorCompactions, HoldTime, AverageFilesPerTablet,
Name]]
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360746 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.Entries=14
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360746 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.EntriesInMemory=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360747 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.Ingest=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360748 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.MajorCompactions=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360748 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.MajorCompactionsQueued=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360749 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.MinorCompactions=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360749 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.MinorCompactionsQueued=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360750 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.OnlineCount=3
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360750 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.OpeningCount=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360751 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.Queries=995
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360751 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.UnopenedCount=0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360752 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.TotalMinorCompactions=33
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360752 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.HoldTime=0.0
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360752 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.AverageFilesPerTablet=0.3333333333333333
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360753 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
Ganglia metric Accumulo.Name=mohit:9997
[27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360753 DEBUG
(com.googlecode.jmxtrans.util.JmxUtils:209) - Finished running
outputWriters for query: Query
[obj=accumulo.server.metrics:service=TServerInfo,name=TabletServerMBean,instance=tserver,
resultAlias=Accumulo, attr=[Entries, EntriesInMemory, Ingest,
MajorCompactions, MajorCompactionsQueued, MinorCompactions,
MinorCompactionsQueued, OnlineCount, OpeningCount, Queries,
UnopenedCount, TotalMinorCompactions, HoldTime, AverageFilesPerTablet,
Name]]
At the ganglia Web UI, I can not see any Accumulo metrices for this
cluster.I have checked every configurations hundreds of times.
If anyone want, I can upload configurations, procedure or any required
info. Please comment on what could be the problem in query execution?
and how can I fix it?
Is it a compatibility issue with Accumulo 1.7.0?
Thanks
Mohit Kaushik
