Re: [ceph-users] Prometheus RADOSGW usage exporter

[Berant Lemmenes]

My apologies, I don't seem to be getting notifications on PRs. I'll review
this week.

Thanks,
Berant

On Mon, Mar 19, 2018 at 5:55 AM, Konstantin Shalygin  wrote:

> Hi Berant
>
>
> I've created prometheus exporter that scrapes the RADOSGW Admin Ops API and
>> exports the usage information for all users and buckets. This is my first
>> prometheus exporter so if anyone has feedback I'd greatly appreciate it.
>> I've tested it against Hammer, and will shortly test against Jewel; though
>> looking at the docs it should work fine for Jewel as well.
>>
>> https://github.com/blemmenes/radosgw_usage_exporter
>>
>
>
> It would be nice, if you take a look on PR's.
>
>
>
>
> k
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Prometheus RADOSGW usage exporter

[Konstantin Shalygin]

Hi Berant



I've created prometheus exporter that scrapes the RADOSGW Admin Ops API and
exports the usage information for all users and buckets. This is my first
prometheus exporter so if anyone has feedback I'd greatly appreciate it.
I've tested it against Hammer, and will shortly test against Jewel; though
looking at the docs it should work fine for Jewel as well.

https://github.com/blemmenes/radosgw_usage_exporter



It would be nice, if you take a look on PR's.




k

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Prometheus RADOSGW usage exporter

[Berant Lemmenes]

Ben,

Thanks for taking a look at it and trying it out! Hmm looks like at some
point where the bucket owner is in the JSON changed. Later in the week I'll
take a look at adding something to take either location into account.

Thanks,
Berant

On Tue, May 30, 2017 at 3:54 AM, Ben Morrice  wrote:

> Hello Berant,
>
> This is very nice! I've had a play with this against our installation of
> Ceph which is Kraken. We had to change the bucket_owner variable to be
> inside the for loop [1] and we are currently not getting any bytes
> sent/received statistics - though this is not an issue with your code, as
> these values are not updated via radosgw-admin either. I think i'm hitting
> this bug http://tracker.ceph.com/issues/19194
>
> [1] for bucket in entry['buckets']:
> print bucket
> bucket_owner = bucket['owner']
>
> Kind regards,
>
> Ben Morrice
>
> __
> Ben Morrice | e: ben.morr...@epfl.ch | t: +41-21-693-9670 
> <+41%2021%20693%2096%2070>
> EPFL / BBP
> Biotech Campus
> Chemin des Mines 9
> 1202 Geneva
> Switzerland
>
> On 25/05/17 16:25, Berant Lemmenes wrote:
>
> Hello all,
>
> I've created prometheus exporter that scrapes the RADOSGW Admin Ops API and
> exports the usage information for all users and buckets. This is my first
> prometheus exporter so if anyone has feedback I'd greatly appreciate it.
> I've tested it against Hammer, and will shortly test against Jewel; though
> looking at the docs it should work fine for Jewel as well.
> https://github.com/blemmenes/radosgw_usage_exporter
>
>
> Sample output:
> radosgw_usage_successful_ops_total{bucket="shard0",category="create_bucket",owner="testuser"}
> 1.0
> radosgw_usage_successful_ops_total{bucket="shard0",category="delete_obj",owner="testuser"}
> 1094978.0
> radosgw_usage_successful_ops_total{bucket="shard0",category="list_bucket",owner="testuser"}
> 2276.0
> radosgw_usage_successful_ops_total{bucket="shard0",category="put_obj",owner="testuser"}
> 1094978.0
> radosgw_usage_successful_ops_total{bucket="shard0",category="stat_bucket",owner="testuser"}
> 20.0
> radosgw_usage_received_bytes_total{bucket="shard0",category="create_bucket",owner="testuser"}
> 0.0
> radosgw_usage_received_bytes_total{bucket="shard0",category="delete_obj",owner="testuser"}
> 0.0
> radosgw_usage_received_bytes_total{bucket="shard0",category="list_bucket",owner="testuser"}
> 0.0
> radosgw_usage_received_bytes_total{bucket="shard0",category="put_obj",owner="testuser"}
> 6352678.0
> radosgw_usage_received_bytes_total{bucket="shard0",category="stat_bucket",owner="testuser"}
> 0.0
> radosgw_usage_sent_bytes_total{bucket="shard0",category="create_bucket",owner="testuser"}
> 19.0
> radosgw_usage_sent_bytes_total{bucket="shard0",category="delete_obj",owner="testuser"}
> 0.0
> radosgw_usage_sent_bytes_total{bucket="shard0",category="list_bucket",owner="testuser"}
> 638339458.0
> radosgw_usage_sent_bytes_total{bucket="shard0",category="put_obj",owner="testuser"}
> 79.0
> radosgw_usage_sent_bytes_total{bucket="shard0",category="stat_bucket",owner="testuser"}
> 380.0
> radosgw_usage_ops_total{bucket="shard0",category="create_bucket",owner="testuser"}
> 1.0
> radosgw_usage_ops_total{bucket="shard0",category="delete_obj",owner="testuser"}
> 1094978.0
> radosgw_usage_ops_total{bucket="shard0",category="list_bucket",owner="testuser"}
> 2276.0
> radosgw_usage_ops_total{bucket="shard0",category="put_obj",owner="testuser"}
> 1094979.0
> radosgw_usage_ops_total{bucket="shard0",category="stat_bucket",owner="testuser"}
> 20.0
>
>
> Thanks,
> Berant
>
>
>
>
> ___
> ceph-users mailing 
> listceph-us...@lists.ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Prometheus RADOSGW usage exporter

[Ben Morrice]

Hello Berant,

This is very nice! I've had a play with this against our installation of 
Ceph which is Kraken. We had to change the bucket_owner variable to be 
inside the for loop [1] and we are currently not getting any bytes 
sent/received statistics - though this is not an issue with your code, 
as these values are not updated via radosgw-admin either. I think i'm 
hitting this bug http://tracker.ceph.com/issues/19194


[1] for bucket in entry['buckets']:
print bucket
bucket_owner = bucket['owner']

Kind regards,

Ben Morrice

__
Ben Morrice | e: ben.morr...@epfl.ch | t: +41-21-693-9670
EPFL / BBP
Biotech Campus
Chemin des Mines 9
1202 Geneva
Switzerland

On 25/05/17 16:25, Berant Lemmenes wrote:

Hello all,

I've created prometheus exporter that scrapes the RADOSGW Admin Ops API and
exports the usage information for all users and buckets. This is my first
prometheus exporter so if anyone has feedback I'd greatly appreciate it.
I've tested it against Hammer, and will shortly test against Jewel; though
looking at the docs it should work fine for Jewel as well.

https://github.com/blemmenes/radosgw_usage_exporter


Sample output:
radosgw_usage_successful_ops_total{bucket="shard0",category="create_bucket",owner="testuser"}
1.0
radosgw_usage_successful_ops_total{bucket="shard0",category="delete_obj",owner="testuser"}
1094978.0
radosgw_usage_successful_ops_total{bucket="shard0",category="list_bucket",owner="testuser"}
2276.0
radosgw_usage_successful_ops_total{bucket="shard0",category="put_obj",owner="testuser"}
1094978.0
radosgw_usage_successful_ops_total{bucket="shard0",category="stat_bucket",owner="testuser"}
20.0
radosgw_usage_received_bytes_total{bucket="shard0",category="create_bucket",owner="testuser"}
0.0
radosgw_usage_received_bytes_total{bucket="shard0",category="delete_obj",owner="testuser"}
0.0
radosgw_usage_received_bytes_total{bucket="shard0",category="list_bucket",owner="testuser"}
0.0
radosgw_usage_received_bytes_total{bucket="shard0",category="put_obj",owner="testuser"}
6352678.0
radosgw_usage_received_bytes_total{bucket="shard0",category="stat_bucket",owner="testuser"}
0.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="create_bucket",owner="testuser"}
19.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="delete_obj",owner="testuser"}
0.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="list_bucket",owner="testuser"}
638339458.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="put_obj",owner="testuser"}
79.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="stat_bucket",owner="testuser"}
380.0
radosgw_usage_ops_total{bucket="shard0",category="create_bucket",owner="testuser"}
1.0
radosgw_usage_ops_total{bucket="shard0",category="delete_obj",owner="testuser"}
1094978.0
radosgw_usage_ops_total{bucket="shard0",category="list_bucket",owner="testuser"}
2276.0
radosgw_usage_ops_total{bucket="shard0",category="put_obj",owner="testuser"}
1094979.0
radosgw_usage_ops_total{bucket="shard0",category="stat_bucket",owner="testuser"}
20.0


Thanks,
Berant



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Prometheus RADOSGW usage exporter

[Berant Lemmenes]

Hello all,

I've created prometheus exporter that scrapes the RADOSGW Admin Ops API and
exports the usage information for all users and buckets. This is my first
prometheus exporter so if anyone has feedback I'd greatly appreciate it.
I've tested it against Hammer, and will shortly test against Jewel; though
looking at the docs it should work fine for Jewel as well.

https://github.com/blemmenes/radosgw_usage_exporter


Sample output:
radosgw_usage_successful_ops_total{bucket="shard0",category="create_bucket",owner="testuser"}
1.0
radosgw_usage_successful_ops_total{bucket="shard0",category="delete_obj",owner="testuser"}
1094978.0
radosgw_usage_successful_ops_total{bucket="shard0",category="list_bucket",owner="testuser"}
2276.0
radosgw_usage_successful_ops_total{bucket="shard0",category="put_obj",owner="testuser"}
1094978.0
radosgw_usage_successful_ops_total{bucket="shard0",category="stat_bucket",owner="testuser"}
20.0
radosgw_usage_received_bytes_total{bucket="shard0",category="create_bucket",owner="testuser"}
0.0
radosgw_usage_received_bytes_total{bucket="shard0",category="delete_obj",owner="testuser"}
0.0
radosgw_usage_received_bytes_total{bucket="shard0",category="list_bucket",owner="testuser"}
0.0
radosgw_usage_received_bytes_total{bucket="shard0",category="put_obj",owner="testuser"}
6352678.0
radosgw_usage_received_bytes_total{bucket="shard0",category="stat_bucket",owner="testuser"}
0.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="create_bucket",owner="testuser"}
19.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="delete_obj",owner="testuser"}
0.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="list_bucket",owner="testuser"}
638339458.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="put_obj",owner="testuser"}
79.0
radosgw_usage_sent_bytes_total{bucket="shard0",category="stat_bucket",owner="testuser"}
380.0
radosgw_usage_ops_total{bucket="shard0",category="create_bucket",owner="testuser"}
1.0
radosgw_usage_ops_total{bucket="shard0",category="delete_obj",owner="testuser"}
1094978.0
radosgw_usage_ops_total{bucket="shard0",category="list_bucket",owner="testuser"}
2276.0
radosgw_usage_ops_total{bucket="shard0",category="put_obj",owner="testuser"}
1094979.0
radosgw_usage_ops_total{bucket="shard0",category="stat_bucket",owner="testuser"}
20.0


Thanks,
Berant
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com