Re: [openstack-dev] [ceilometer]Can't find meter anywhere with ceilometer post REST API

2017-03-21 Thread Hui Xiang 
Yurii,

   Thanks, with the config listed above, it works now.

Hui.

On Tue, Mar 21, 2017 at 2:50 PM, Yurii Prokulevych 
wrote:

> Pipeline's config looks good. Could U please enable debug/verbose in
> ceilometer.conf and check ceilometer/collector.log ?
>
> ---
> Yurii
>
> On Tue, 2017-03-21 at 11:40 +0800, Hui Xiang wrote:
> > Thanks gordon for your info.
> >
> > The reason why not using gnocchi in mitaka is that we are using
> > collectd-ceilometer-plugin[1] to posting samples to ceilometer
> > through ceilometer-api, after mitaka yes we will all move to gnocchi.
> >
> >
> > """
> > when posting samples to ceilometer-api, the data goes through
> > pipeline before being stored. therefore, you need notification-agent
> > enabled AND you need to make sure the pipeline.yaml accepts the
> > meter.
> > """
> > As the samples posted doesn't have event_type, so I guess you mean I
> > don't need to edit the event_pipeline.yaml, but need to edit the
> > pipeline.yaml to accepts the meter. Could you kindly check whether
> > below simple example make sense to accept the meter?  Does the source
> > name need to match the source field in the sample or it can be
> > defined as anyone.
> >
> > > [{"counter_name": "interface.if_errors",
> > >   "user_id": "5457b977c25e4498a31a3c1c78829631",
> > >   "resource_id": "localhost-ovs-system",
> > >   "timestamp": "2017-03-17T02:26:46",
> > >   "resource_metadata": {},
> > >   "source": "5b1525a8eb2d4739a83b296682aed023:collectd",
> > >   "counter_unit": "Errors/s",
> > >   "counter_volume": 0.0,
> > >   "project_id": "5b1525a8eb2d4739a83b296682aed023",
> > >   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
> > >   "counter_type": "delta"},
> > >
> >
> >
> > sources:
> > - name: meter_source
> >   interval: 60
> >   meters:
> >   - "interface.if_errors"
> >   sinks:
> >   - meter_sink
> >
> > sinks:
> > - name: meter_sink
> >   transformers:
> >   publishers:
> >   - notifier://
> >
> >
> > Does the source name need to matching the source field in the sample
> > or it can be defined as any.
> >
> > [1]. https://github.com/openstack/collectd-ceilometer-plugin
> >
> >
> > Thanks.
> > Hui.
> >
> >
> > On Tue, Mar 21, 2017 at 4:21 AM, gordon chung  wrote:
> > >
> > >
> > > On 18/03/17 04:54 AM, Hui Xiang wrote:
> > > > Hi folks,
> > > >
> > > >   I am trying to post samples from third part software to
> > > ceilometer via
> > > > the REST API as below with Mitaka version. I can see ceilometer-
> > > api has
> > > > received this post, and seems forwarded to ceilometer
> > > notification agent
> > > > through RMQ.
> > > >
> > >
> > > first and most importantly, the ceilometer-api is deprecated and
> > > not
> > > supported upstream anymore. please use gnocchi for proper time
> > > series
> > > storage (or whatever storage solution you feel comfortable with)
> > >
> > > >
> > > > 2. LOG
> > > > 56:17] "*POST /v2/meters/interface.if_packets HTTP/1.1*" 201 -
> > > > 2017-03-17 16:56:17.378 52955 DEBUG
> > > oslo_messaging._drivers.amqpdriver
> > > > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> > > > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023
> > > - - -]
> > > > CAST unique_id: 64a6bae3bbcc4b7dab4dceb13cf7f81b NOTIFY exchange
> > > > 'ceilometer' topic 'notifications.sample' _send
> > > > /usr/lib/python2.7/site-
> > > packages/oslo_messaging/_drivers/amqpdriver.py:438
> > > > 2017-03-17 16:56:17.382 52955 INFO werkzeug
> > > > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> > > > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023
> > > - - -]
> > > > 192.168.0.3 - - [17/Mar/2017
> > > >
> > > >
> > > > 3. REST API return result
> > > > [{"counter_name": "interface.if_errors",
> > > >   "user_id": "5457b977c25e4498a31a3c1c78829631",
> > > >   "resource_id": "localhost-ovs-system",
> > > >   "timestamp": "2017-03-17T02:26:46",
> > > >   "resource_metadata": {},
> > > >   "source": "5b1525a8eb2d4739a83b296682aed023:collectd",
> > > >   "counter_unit": "Errors/s",
> > > >   "counter_volume": 0.0,
> > > >   "project_id": "5b1525a8eb2d4739a83b296682aed023",
> > > >   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
> > > >   "counter_type": "delta"},
> > > >
> > >
> > > when posting samples to ceilometer-api, the data goes through
> > > pipeline
> > > before being stored. therefore, you need notification-agent enabled
> > > AND
> > > you need to make sure the pipeline.yaml accepts the meter.
> > >
> > > --
> > > gord
> > >
> > > ___
> > > ___
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu
> > > bscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> >
> > _
> > _
> > OpenStack Development Mailing

Re: [openstack-dev] [ceilometer]Can't find meter anywhere with ceilometer post REST API

2017-03-21 Thread Hui Xiang 
gordon,

  Thanks much, it works by adding below section in pipeline.yaml

- name: collectd_source
  interval: 60
  meters:
  - "load.load"
  - "memory.memory"
  - "interface.if_dropped"
  - "interface.if_errors"

- name: collectd_sink
  transformers:
  publishers:
  - notifier://


   Another question, what is the difference between ceilometer backend
database with the database:// configured in publishers? If I didn't set
'backend=xx' in ceilometer.conf but set
'connection=mongodb://user:password@ip_address/ceilometer', seems it
connected to mongodb other than any backend options(sqlalchemy, mysql,etc),
does the database:// configured in publishers should be actually
'mysql://user:password@ip_address/ceilometer' ?


Thanks.
Hui.



On Tue, Mar 21, 2017 at 11:40 AM, Hui Xiang  wrote:

> Thanks gordon for your info.
>
> The reason why not using gnocchi in mitaka is that we are using
> collectd-ceilometer-plugin[1] to posting samples to ceilometer through
> ceilometer-api, after mitaka yes we will all move to gnocchi.
>
>
> """
> when posting samples to ceilometer-api, the data goes through
> pipeline before being stored. therefore, you need notification-agent
> enabled AND you need to make sure the pipeline.yaml accepts the meter.
> """
> As the samples posted doesn't have event_type, so I guess you mean I don't
> need to edit the event_pipeline.yaml, but need to edit the pipeline.yaml to
> accepts the meter. Could you kindly check whether below simple example make
> sense to accept the meter?  Does the source name need to match the source
> field in the sample or it can be defined as anyone.
>
> > [{"counter_name": "interface.if_errors",
> >   "user_id": "5457b977c25e4498a31a3c1c78829631",
> >   "resource_id": "localhost-ovs-system",
> >   "timestamp": "2017-03-17T02:26:46",
> >   "resource_metadata": {},
> >   "source": *"5b1525a8eb2d4739a83b296682aed023:collectd*",
> >   "counter_unit": "Errors/s",
> >   "counter_volume": 0.0,
> >   "project_id": "5b1525a8eb2d4739a83b296682aed023",
> >   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
> >   "counter_type": "delta"},
> >
>
>
> sources:
> - name: *meter_source*
>   interval: 60
>   meters:
>   - "interface.if_errors"
>   sinks:
>   - meter_sink
>
> sinks:
> - name: meter_sink
>   transformers:
>   publishers:
>   - notifier://
>
>
> Does the source name need to matching the source field in the sample or it
> can be defined as any.
>
> [1]. https://github.com/openstack/collectd-ceilometer-plugin
>
>
> Thanks.
> Hui.
>
>
> On Tue, Mar 21, 2017 at 4:21 AM, gordon chung  wrote:
>
>>
>>
>> On 18/03/17 04:54 AM, Hui Xiang wrote:
>> > Hi folks,
>> >
>> >   I am trying to post samples from third part software to ceilometer via
>> > the REST API as below with Mitaka version. I can see ceilometer-api has
>> > received this post, and seems forwarded to ceilometer notification agent
>> > through RMQ.
>> >
>>
>> first and most importantly, the ceilometer-api is deprecated and not
>> supported upstream anymore. please use gnocchi for proper time series
>> storage (or whatever storage solution you feel comfortable with)
>>
>> >
>> > 2. LOG
>> > 56:17] "*POST /v2/meters/interface.if_packets HTTP/1.1*" 201 -
>> > 2017-03-17 16:56:17.378 52955 DEBUG oslo_messaging._drivers.amqpdriver
>> > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
>> > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023 - -
>> -]
>> > CAST unique_id: 64a6bae3bbcc4b7dab4dceb13cf7f81b NOTIFY exchange
>> > 'ceilometer' topic 'notifications.sample' _send
>> > /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/
>> amqpdriver.py:438
>> > 2017-03-17 16:56:17.382 52955 INFO werkzeug
>> > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
>> > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023 - -
>> -]
>> > 192.168.0.3 - - [17/Mar/2017
>> >
>> >
>> > 3. REST API return result
>> > [{"counter_name": "interface.if_errors",
>> >   "user_id": "5457b977c25e4498a31a3c1c78829631",
>> >   "resource_id": "localhost-ovs-system",
>> >   "timestamp": "2017-03-17T02:26:46",
>> >   "resource_metadata": {},
>> >   "source": "5b1525a8eb2d4739a83b296682aed023:collectd",
>> >   "counter_unit": "Errors/s",
>> >   "counter_volume": 0.0,
>> >   "project_id": "5b1525a8eb2d4739a83b296682aed023",
>> >   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
>> >   "counter_type": "delta"},
>> >
>>
>> when posting samples to ceilometer-api, the data goes through pipeline
>> before being stored. therefore, you need notification-agent enabled AND
>> you need to make sure the pipeline.yaml accepts the meter.
>>
>> --
>> gord
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>>

Re: [openstack-dev] [ceilometer]Can't find meter anywhere with ceilometer post REST API

2017-03-21 Thread Yurii Prokulevych 
Pipeline's config looks good. Could U please enable debug/verbose in
ceilometer.conf and check ceilometer/collector.log ?

---
Yurii

On Tue, 2017-03-21 at 11:40 +0800, Hui Xiang wrote:
> Thanks gordon for your info.
> 
> The reason why not using gnocchi in mitaka is that we are using
> collectd-ceilometer-plugin[1] to posting samples to ceilometer
> through ceilometer-api, after mitaka yes we will all move to gnocchi.
> 
> 
> """
> when posting samples to ceilometer-api, the data goes through
> pipeline before being stored. therefore, you need notification-agent
> enabled AND you need to make sure the pipeline.yaml accepts the
> meter.
> """
> As the samples posted doesn't have event_type, so I guess you mean I
> don't need to edit the event_pipeline.yaml, but need to edit the
> pipeline.yaml to accepts the meter. Could you kindly check whether
> below simple example make sense to accept the meter?  Does the source
> name need to match the source field in the sample or it can be
> defined as anyone. 
> 
> > [{"counter_name": "interface.if_errors",
> >   "user_id": "5457b977c25e4498a31a3c1c78829631",
> >   "resource_id": "localhost-ovs-system",
> >   "timestamp": "2017-03-17T02:26:46",
> >   "resource_metadata": {},
> >   "source": "5b1525a8eb2d4739a83b296682aed023:collectd",
> >   "counter_unit": "Errors/s",
> >   "counter_volume": 0.0,
> >   "project_id": "5b1525a8eb2d4739a83b296682aed023",
> >   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
> >   "counter_type": "delta"},
> >
> 
> 
> sources:
>     - name: meter_source
>       interval: 60
>       meters:
>           - "interface.if_errors"
>       sinks:
>           - meter_sink
> 
> sinks:
>     - name: meter_sink
>       transformers:
>       publishers:
>           - notifier://
> 
> 
> Does the source name need to matching the source field in the sample
> or it can be defined as any.
> 
> [1]. https://github.com/openstack/collectd-ceilometer-plugin
> 
> 
> Thanks.
> Hui.
> 
> 
> On Tue, Mar 21, 2017 at 4:21 AM, gordon chung  wrote:
> > 
> > 
> > On 18/03/17 04:54 AM, Hui Xiang wrote:
> > > Hi folks,
> > >
> > >   I am trying to post samples from third part software to
> > ceilometer via
> > > the REST API as below with Mitaka version. I can see ceilometer-
> > api has
> > > received this post, and seems forwarded to ceilometer
> > notification agent
> > > through RMQ.
> > >
> > 
> > first and most importantly, the ceilometer-api is deprecated and
> > not
> > supported upstream anymore. please use gnocchi for proper time
> > series
> > storage (or whatever storage solution you feel comfortable with)
> > 
> > >
> > > 2. LOG
> > > 56:17] "*POST /v2/meters/interface.if_packets HTTP/1.1*" 201 -
> > > 2017-03-17 16:56:17.378 52955 DEBUG
> > oslo_messaging._drivers.amqpdriver
> > > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> > > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023
> > - - -]
> > > CAST unique_id: 64a6bae3bbcc4b7dab4dceb13cf7f81b NOTIFY exchange
> > > 'ceilometer' topic 'notifications.sample' _send
> > > /usr/lib/python2.7/site-
> > packages/oslo_messaging/_drivers/amqpdriver.py:438
> > > 2017-03-17 16:56:17.382 52955 INFO werkzeug
> > > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> > > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023
> > - - -]
> > > 192.168.0.3 - - [17/Mar/2017
> > >
> > >
> > > 3. REST API return result
> > > [{"counter_name": "interface.if_errors",
> > >   "user_id": "5457b977c25e4498a31a3c1c78829631",
> > >   "resource_id": "localhost-ovs-system",
> > >   "timestamp": "2017-03-17T02:26:46",
> > >   "resource_metadata": {},
> > >   "source": "5b1525a8eb2d4739a83b296682aed023:collectd",
> > >   "counter_unit": "Errors/s",
> > >   "counter_volume": 0.0,
> > >   "project_id": "5b1525a8eb2d4739a83b296682aed023",
> > >   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
> > >   "counter_type": "delta"},
> > >
> > 
> > when posting samples to ceilometer-api, the data goes through
> > pipeline
> > before being stored. therefore, you need notification-agent enabled
> > AND
> > you need to make sure the pipeline.yaml accepts the meter.
> > 
> > --
> > gord
> > 
> > ___
> > ___
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu
> > bscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [ceilometer]Can't find meter anywhere with ceilometer post REST API

2017-03-20 Thread Hui Xiang 
Thanks gordon for your info.

The reason why not using gnocchi in mitaka is that we are using
collectd-ceilometer-plugin[1] to posting samples to ceilometer through
ceilometer-api, after mitaka yes we will all move to gnocchi.


"""
when posting samples to ceilometer-api, the data goes through
pipeline before being stored. therefore, you need notification-agent
enabled AND you need to make sure the pipeline.yaml accepts the meter.
"""
As the samples posted doesn't have event_type, so I guess you mean I don't
need to edit the event_pipeline.yaml, but need to edit the pipeline.yaml to
accepts the meter. Could you kindly check whether below simple example make
sense to accept the meter?  Does the source name need to match the source
field in the sample or it can be defined as anyone.

> [{"counter_name": "interface.if_errors",
>   "user_id": "5457b977c25e4498a31a3c1c78829631",
>   "resource_id": "localhost-ovs-system",
>   "timestamp": "2017-03-17T02:26:46",
>   "resource_metadata": {},
>   "source": *"5b1525a8eb2d4739a83b296682aed023:collectd*",
>   "counter_unit": "Errors/s",
>   "counter_volume": 0.0,
>   "project_id": "5b1525a8eb2d4739a83b296682aed023",
>   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
>   "counter_type": "delta"},
>


sources:
- name: *meter_source*
  interval: 60
  meters:
  - "interface.if_errors"
  sinks:
  - meter_sink

sinks:
- name: meter_sink
  transformers:
  publishers:
  - notifier://


Does the source name need to matching the source field in the sample or it
can be defined as any.

[1]. https://github.com/openstack/collectd-ceilometer-plugin


Thanks.
Hui.


On Tue, Mar 21, 2017 at 4:21 AM, gordon chung  wrote:

>
>
> On 18/03/17 04:54 AM, Hui Xiang wrote:
> > Hi folks,
> >
> >   I am trying to post samples from third part software to ceilometer via
> > the REST API as below with Mitaka version. I can see ceilometer-api has
> > received this post, and seems forwarded to ceilometer notification agent
> > through RMQ.
> >
>
> first and most importantly, the ceilometer-api is deprecated and not
> supported upstream anymore. please use gnocchi for proper time series
> storage (or whatever storage solution you feel comfortable with)
>
> >
> > 2. LOG
> > 56:17] "*POST /v2/meters/interface.if_packets HTTP/1.1*" 201 -
> > 2017-03-17 16:56:17.378 52955 DEBUG oslo_messaging._drivers.amqpdriver
> > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023 - - -]
> > CAST unique_id: 64a6bae3bbcc4b7dab4dceb13cf7f81b NOTIFY exchange
> > 'ceilometer' topic 'notifications.sample' _send
> > /usr/lib/python2.7/site-packages/oslo_messaging/_
> drivers/amqpdriver.py:438
> > 2017-03-17 16:56:17.382 52955 INFO werkzeug
> > [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> > 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023 - - -]
> > 192.168.0.3 - - [17/Mar/2017
> >
> >
> > 3. REST API return result
> > [{"counter_name": "interface.if_errors",
> >   "user_id": "5457b977c25e4498a31a3c1c78829631",
> >   "resource_id": "localhost-ovs-system",
> >   "timestamp": "2017-03-17T02:26:46",
> >   "resource_metadata": {},
> >   "source": "5b1525a8eb2d4739a83b296682aed023:collectd",
> >   "counter_unit": "Errors/s",
> >   "counter_volume": 0.0,
> >   "project_id": "5b1525a8eb2d4739a83b296682aed023",
> >   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
> >   "counter_type": "delta"},
> >
>
> when posting samples to ceilometer-api, the data goes through pipeline
> before being stored. therefore, you need notification-agent enabled AND
> you need to make sure the pipeline.yaml accepts the meter.
>
> --
> gord
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer]Can't find meter anywhere with ceilometer post REST API

2017-03-20 Thread gordon chung 


On 18/03/17 04:54 AM, Hui Xiang wrote:
> Hi folks,
>
>   I am trying to post samples from third part software to ceilometer via
> the REST API as below with Mitaka version. I can see ceilometer-api has
> received this post, and seems forwarded to ceilometer notification agent
> through RMQ.
>

first and most importantly, the ceilometer-api is deprecated and not 
supported upstream anymore. please use gnocchi for proper time series 
storage (or whatever storage solution you feel comfortable with)

>
> 2. LOG
> 56:17] "*POST /v2/meters/interface.if_packets HTTP/1.1*" 201 -
> 2017-03-17 16:56:17.378 52955 DEBUG oslo_messaging._drivers.amqpdriver
> [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023 - - -]
> CAST unique_id: 64a6bae3bbcc4b7dab4dceb13cf7f81b NOTIFY exchange
> 'ceilometer' topic 'notifications.sample' _send
> /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:438
> 2017-03-17 16:56:17.382 52955 INFO werkzeug
> [req-1c4ea84d-ea53-4518-81ea-6c0bffa9745d
> 5457b977c25e4498a31a3c1c78829631 5b1525a8eb2d4739a83b296682aed023 - - -]
> 192.168.0.3 - - [17/Mar/2017
>
>
> 3. REST API return result
> [{"counter_name": "interface.if_errors",
>   "user_id": "5457b977c25e4498a31a3c1c78829631",
>   "resource_id": "localhost-ovs-system",
>   "timestamp": "2017-03-17T02:26:46",
>   "resource_metadata": {},
>   "source": "5b1525a8eb2d4739a83b296682aed023:collectd",
>   "counter_unit": "Errors/s",
>   "counter_volume": 0.0,
>   "project_id": "5b1525a8eb2d4739a83b296682aed023",
>   "message_id": "2b4ce294-0ab9-11e7-8058-026ea687824d",
>   "counter_type": "delta"},
>

when posting samples to ceilometer-api, the data goes through pipeline 
before being stored. therefore, you need notification-agent enabled AND 
you need to make sure the pipeline.yaml accepts the meter.

-- 
gord

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev