Question #290796 on Graphite changed:
https://answers.launchpad.net/graphite/+question/290796
Description changed to:
Hi,
I'm testing the aggregator and something is unexpected.
Here is my aggregation rule:
<env>.<system>.<host>.<app>.<metrics>.sum_10 (10) = sum
<env>.<system>.<host>.<app>.<<metrics>>.raw$
Now I've added this line in carbon/clients.py:321
def sendDatapoint(self, metric, datapoint):
for destination in self.router.getDestinations(metric):
-> log.clients("Sending metric to destination %s" % (metric))
self.client_factories[destination].sendDatapoint(metric, datapoint)
and carbon/aggregator/buffer.py:69:
state.events.metricGenerated(self.metric_path, datapoint)
-> log.aggregator("Metric generated %s" % self.metric_path)
state.instrumentation.increment('aggregateDatapointsSent')
I'm sending these 2 every 5 secs (in python):
"test.bash.stats", random.randint(0,100)
"TEST.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw",
random.randint(0,100)
And here are the logs when starting my aggregator in DEBUG:
-------------------------------------------------
13/04/2016 15:11:36 :: [console] Log opened.
13/04/2016 15:11:36 :: [console] twistd 16.0.0 (/usr/bin/python 2.7.6) starting
up.
13/04/2016 15:11:36 :: [console] reactor class:
twisted.internet.epollreactor.EPollReactor.
13/04/2016 15:11:36 :: [console] CarbonReceiverFactory starting on 2023
13/04/2016 15:11:36 :: [console] Starting factory
<carbon.service.CarbonReceiverFactory instance at 0x7f1533a7c908>
13/04/2016 15:11:36 :: [console] CarbonReceiverFactory starting on 2024
13/04/2016 15:11:36 :: [console] Starting factory
<carbon.service.CarbonReceiverFactory instance at 0x7f1533a7c7e8>
13/04/2016 15:11:36 :: [console] Starting factory
CarbonClientFactory(10.33.21.38:2014:None)
13/04/2016 15:11:36 :: [clients]
CarbonClientFactory(XX.XX.XX.XX:2014:None)::startedConnecting (XX.XX.XX.XX:2014)
13/04/2016 15:11:36 :: [clients]
CarbonClientProtocol(XX.XX.XX.XX::2014:None)::connectionMade
13/04/2016 15:12:23 :: [listener] MetricPickleReceiver connection with
YY.YY.YY.YY:55986 established
13/04/2016 15:12:23 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:23 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:23 :: [aggregator] Allocating new metric buffer for
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:12:23 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
13/04/2016 15:12:28 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:28 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:28 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
13/04/2016 15:12:33 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:33 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:33 :: [console] Couldn't match metric
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10 with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:33 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:12:33 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:12:33 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
13/04/2016 15:12:38 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:38 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:38 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
13/04/2016 15:12:43 :: [console] Couldn't match metric
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10 with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:43 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:12:43 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:12:43 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:43 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:43 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
13/04/2016 15:12:48 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:48 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:48 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
13/04/2016 15:12:53 :: [console] Couldn't match metric
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10 with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:53 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:12:53 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:12:53 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:53 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:53 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
13/04/2016 15:12:58 :: [console] Couldn't match metric test.bash.stats with any
aggregation rule. Passing on un-aggregated.
13/04/2016 15:12:58 :: [clients] Sending metric to destination test.bash.stats
13/04/2016 15:12:58 :: [clients] Sending metric to destination
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.raw
-------------------------------------------------
1. raw data is being passed over. Isn't the aggregator supposed to retain these
values and pass them?
2. This line shows that the output of an aggregation is being re-aggregated:
13/04/2016 15:12:53 :: [console] Couldn't match metric
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10 with any
aggregation rule. Passing on un-aggregated.
So I want to send the least amount of data possible, so just the aggregated
output calculated every X seconds. I've seen the config item: FORWARD_ALL = True
But if I switch it to False, here are the logs:
-------------------------------------------------
13/04/2016 15:26:46 :: [console] twistd 16.0.0 (/usr/bin/python 2.7.6) starting
up.
13/04/2016 15:26:46 :: [console] reactor class:
twisted.internet.epollreactor.EPollReactor.
13/04/2016 15:26:46 :: [console] CarbonReceiverFactory starting on 2023
13/04/2016 15:26:46 :: [console] Starting factory
<carbon.service.CarbonReceiverFactory instance at 0x7f74fdf1f758>
13/04/2016 15:26:46 :: [console] CarbonReceiverFactory starting on 2024
13/04/2016 15:26:46 :: [console] Starting factory
<carbon.service.CarbonReceiverFactory instance at 0x7f74fdf1f638>
13/04/2016 15:26:46 :: [console] Starting factory
CarbonClientFactory(10.33.21.38:2014:None)
13/04/2016 15:26:46 :: [clients]
CarbonClientFactory(XX.XX.XX.XX::2014:None)::startedConnecting
(XX.XX.XX.XX::2014)
13/04/2016 15:26:46 :: [clients]
CarbonClientProtocol(XX.XX.XX.XX::2014:None)::connectionMade
13/04/2016 15:26:52 :: [listener] MetricPickleReceiver connection with
YY.YY.YY.YY:57884 established
13/04/2016 15:26:52 :: [aggregator] Allocating new metric buffer for
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:27:02 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:27:12 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:27:22 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:27:32 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:27:42 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:27:52 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
13/04/2016 15:28:02 :: [aggregator] Metric generated
LIVE.boxA.something.applicationXX.QUERY_TIMES.ODB.CURSSQL.sum_10
-------------------------------------------------
As you can see nothing is being sent. Not the test.bash.stats, nor .raw, nor
.sum10
Is there something I am doing wrong or I didn't understand the purpose of the
aggregator?
Regards
Francois
--
You received this question notification because your team graphite-dev
is an answer contact for Graphite.
_______________________________________________
Mailing list: https://launchpad.net/~graphite-dev
Post to : graphite-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~graphite-dev
More help : https://help.launchpad.net/ListHelp
