Re: [openstack-dev] [ceilometer][oslo.cache][oslo] ceilometer fails to publish to gnocchi

[Ben Nemec]

On 11/6/18 3:25 AM, Eyal B wrote:

Hi,
Because of this fix https://review.openstack.org/#/c/611369/ ceilometer 
which uses oslo.cache for redis fails to publish to gnocchi


see this log: 
http://logs.openstack.org/15/615415/1/check/vitrage-dsvm-datasources-py27/8d9e39e/logs/screen-ceilometer-anotification.txt.gz#_Nov_04_13_12_28_656863


Hmm, looks like the problem is that a memached-specific fix is also 
affecting the redis driver. We probably need to blacklist this release 
until we can come up with a fix: https://review.openstack.org/615935


I've opened a bug against oslo.cache as well: 
https://bugs.launchpad.net/oslo.cache/+bug/1801967


-Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][oslo.cache] ceilometer fails to publish to gnocchi

[Eyal B]

Hi,
Because of this fix https://review.openstack.org/#/c/611369/ ceilometer
which uses oslo.cache for redis fails to publish to gnocchi

see this log:
http://logs.openstack.org/15/615415/1/check/vitrage-dsvm-datasources-py27/8d9e39e/logs/screen-ceilometer-anotification.txt.gz#_Nov_04_13_12_28_656863

BR
  Eyal
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] ceilometer-powervm 7.0.0.0rc1 (rocky)

[no-reply]

Hello everyone,

A new release candidate for ceilometer-powervm for the end of the Rocky
cycle is available!  You can find the source code tarball at:

https://tarballs.openstack.org/ceilometer-powervm/

Unless release-critical issues are found that warrant a release
candidate respin, this candidate will be formally released as the
final Rocky release. You are therefore strongly
encouraged to test and validate this tarball!

Alternatively, you can directly test the stable/rocky release
branch at:


http://git.openstack.org/cgit/openstack/ceilometer-powervm/log/?h=stable/rocky

Release notes for ceilometer-powervm can be found at:

http://docs.openstack.org/releasenotes/ceilometer-powervm/




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][panko][pike] elasticsearch integration

[cristian.calin]

ilometer.pipeline  >>>>>return _inner()
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>  File 
"/opt/ceilometer/lib/python3.5/site-packages/tenacity/__init__.py", line 171, 
in wrapped_f
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>return 
self.call(f, *args, **kw)
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>  File 
"/opt/ceilometer/lib/python3.5/site-packages/tenacity/__init__.py", line 248, 
in call
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>
start_time=start_time)
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>  File 
"/opt/ceilometer/lib/python3.5/site-packages/tenacity/__init__.py", line 217, 
in iter
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>
six.raise_from(RetryError(fut), fut.exception())
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>  File "", 
line 2, in raise_from
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>tenacity.RetryError: 
RetryError[]
2018-06-20 07:06:11.537 24 TRACE ceilometer.pipeline  >>>>>

I changed my panko.conf to:
[database]
connection = es://user:password@elasticsearch.service.consul.:9200
es_ssl_enable = False
[storage]
es_index_name = events

But I get the same error which means that the es_* parameters are not properly 
merged from panko.conf when ceilometer-agent-notification starts up.

From: cristian.ca...@orange.com [mailto:cristian.ca...@orange.com]
Sent: Wednesday, June 20, 2018 9:44 AM
To: openstack-operat...@lists.openstack.org
Cc: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [ceilometer][panko][pike] elasticsearch integration

Hello,


I'm trying to run ceilometer with panko publishers in pike release and when I 
run the ceilometer-agent-notification I get a trace complaining about 
NoSuchOptError, but without the actual parameter that is missing (see trace 
below).

I have configured panko.conf with the following:

[database]
connection = es://user:password@elasticsearch.service.consul.:9200
[storage]
es_ssl_enable = False
es_index_name = events


As far as I  can tell from the debug log, the storage.es_ssl_enable and 
storage.es_index_name parameters are not loaded, they don't show up in the 
"cotyledon.oslo_config_glue" output so I assume the trace relates to these 
parameters. Has anybody else seen this error before?

PS: sorry for CC'ing the dev list but I hope to reach the right audience
 TRACE 
{"asctime": "2018-06-20 05:49:09.405","process": "59","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:10.436","process": "61","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:11.409","process": "63","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:18.467","process": "57","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:18.468","process": "57","levelname": 
"ERROR","name": "ceilometer.pipeline", "instance": {},"message":"Unable to load 
publisher panko://"}: RetryError: RetryError[]
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >>>&g

[openstack-dev] [ceilometer][panko][pike] elasticsearch integration

[cristian.calin]

Hello,


I'm trying to run ceilometer with panko publishers in pike release and when I 
run the ceilometer-agent-notification I get a trace complaining about 
NoSuchOptError, but without the actual parameter that is missing (see trace 
below).

I have configured panko.conf with the following:

[database]
connection = es://user:password@elasticsearch.service.consul.:9200
[storage]
es_ssl_enable = False
es_index_name = events


As far as I  can tell from the debug log, the storage.es_ssl_enable and 
storage.es_index_name parameters are not loaded, they don't show up in the 
"cotyledon.oslo_config_glue" output so I assume the trace relates to these 
parameters. Has anybody else seen this error before?

PS: sorry for CC'ing the dev list but I hope to reach the right audience
 TRACE 
{"asctime": "2018-06-20 05:49:09.405","process": "59","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:10.436","process": "61","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:11.409","process": "63","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:18.467","process": "57","levelname": 
"DEBUG","name": "panko.storage", "instance": {},"message":"looking for 'es' 
driver in panko.storage"} {"funcName": "get_connection","source": {"p
ath": 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py","lineno":
 "84"}}
{"asctime": "2018-06-20 05:49:18.468","process": "57","levelname": 
"ERROR","name": "ceilometer.pipeline", "instance": {},"message":"Unable to load 
publisher panko://"}: RetryError: RetryError[]
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >Traceback (most 
recent call last):
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/ceilometer/pipeline.py", line 419, 
in __init__
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >
self.publishers.append(publisher_manager.get(p))
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/ceilometer/pipeline.py", line 713, 
in get
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >
'ceilometer.%s.publisher' % self._purpose)
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/ceilometer/publisher/__init__.py", 
line 36, in get_publisher
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >return 
loaded_driver.driver(parse_result)
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/panko/publisher/database.py", line 
35, in __init__
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >self.conn = 
storage.get_connection_from_config(conf)
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/panko/storage/__init__.py", line 
73, in get_connection_from_config
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >return _inner()
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/tenacity/__init__.py", line 171, 
in wrapped_f
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >return 
self.call(f, *args, **kw)
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/tenacity/__init__.py", line 248, 
in call
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >
start_time=start_time)
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/tenacity/__init__.py", line 217, 
in iter
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >
six.raise_from(RetryError(fut), fut.exception())
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >  File 
"/opt/ceilometer/lib/python2.7/site-packages/six.py", line 718, in raise_from
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >raise value
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >RetryError: 
RetryError[]
2018-06-20 05:49:18.468 57 TRACE ceilometer.pipeline  >

_

Ce message et ses pieces 

[openstack-dev] [ceilometer] Ceilometer-file-publisher-compression-csv-format

[Kwan, Louie]

Submitted the following review on April 19, 

https://review.openstack.org/#/c/562768/

Would like to know who else could be on the reviewer list and anything else for 
the next step?

Thanks.
Louie
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] [gnocchi] keystone verification failed.

[Julien Danjou]

On Tue, Mar 20 2018, __ mango. wrote:

> hi，
>  I have a question about the validation of gnocchi keystone.
>  I run the following command, but it is not successful.(api.auth_mode :basic, 
> basic mode can be 
>  # gnocchi status --debug
> REQ: curl -g -i -X GET http://localhost:8041/v1/status?details=False -H
> "Authorization: {SHA1}d4daf1cf567f14f32dbc762154b3a281b4ea4c62" -H "Accept:
> application/json, */*" -H "User-Agent: gnocchi keystoneauth1/3.1.0
> python-requests/2.18.1 CPython/2.7.12"
> Starting new HTTP connection (1): localhost
> http://localhost:8041 "GET /v1/status?details=False HTTP/1.1" 401 114
> RESP: [401] Content-Type: application/json Content-Length: 114 
> WWW-Authenticate: Keystone uri='http://controller:5000/v3' Connection: 
> Keep-Alive 
> RESP BODY: {"error": {"message": "The request you have made requires 
> authentication.", "code": 401, "title": "Unauthorized"}}
> The request you have made requires authentication. (HTTP 401)

You need to be authed as "admin" to get the status.

-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] [gnocchi] keystone verification failed.

[__ mango.]

hi??
 I have a question about the validation of gnocchi keystone.
 I run the following command, but it is not successful.(api.auth_mode :basic, 
basic mode can be 
 # gnocchi status --debug
REQ: curl -g -i -X GET http://localhost:8041/v1/status?details=False -H 
"Authorization: {SHA1}d4daf1cf567f14f32dbc762154b3a281b4ea4c62" -H "Accept: 
application/json, */*" -H "User-Agent: gnocchi keystoneauth1/3.1.0 
python-requests/2.18.1 CPython/2.7.12"
Starting new HTTP connection (1): localhost
http://localhost:8041 "GET /v1/status?details=False HTTP/1.1" 401 114
RESP: [401] Content-Type: application/json Content-Length: 114 
WWW-Authenticate: Keystone uri='http://controller:5000/v3' Connection: 
Keep-Alive 
RESP BODY: {"error": {"message": "The request you have made requires 
authentication.", "code": 401, "title": "Unauthorized"}}
The request you have made requires authentication. (HTTP 401)

Please help me, thank you very much.

Ps:
I have configured the following__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][gnocchi]-gnocchi installation failed.

[__ mango.]

hi??
I refer to: 
https://docs.openstack.org/ceilometer/pike/install/install-base-ubuntu.html to 
install gnocchi,
Operation (apt-get installed gnocchi-api gnocchi-gnocchiclient),
When I tried to launch the gnocc-api, I got the message.
No gnocchi- API starts. Service :gnocchi-api unit. The service was not found.
I checked /etc/init.d and no script gnocchi-api(although gnocchi-metricd is, 
and it works).
Why is that? Please help me to solve this problem. Thank you very much.__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[gordon chung]

hey Dan,

On 2018-01-11 06:05 PM, Daniel Dyer wrote:
> My understanding was that the API is not officially deprecated until queens. 
> Is this not the case?

not quite. we removed the API permanently in queens. it was actually 
deprecated back in 2016[1] officially. we've unofficially/transparently 
been telling people to switch to Gnocchi (or whatever target you want to 
put into publisher) for longer than that as the legacy api/storage 
hasn't been touched meaningfully since 2015.

given the realities of the project and OpenStack, our project was just 
more proactive in culling stale and/or unusable code.

as it stands, ceilometer solely generates/normalises data about 
openstack resources and publishes data to consumers. other services are 
required to leverage and add value to the data.

[1] 
http://lists.openstack.org/pipermail/openstack-dev/2016-October/105042.html

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Emilien Macchi]

On Thu, Jan 11, 2018 at 3:05 PM, Daniel Dyer  wrote:
> My understanding was that the API is not officially deprecated until queens. 
> Is this not the case?

https://docs.openstack.org/releasenotes/ceilometer/ocata.html#deprecation-notes
https://docs.openstack.org/releasenotes/ceilometer/unreleased.html#upgrade-notes
(queens)

Ceilometer API was deprecated in Ocata and removed in Queens.
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Daniel Dyer]

Gordon,

My understanding was that the API is not officially deprecated until queens. Is 
this not the case?

Dan

> On Jan 11, 2018, at 7:05 AM, gordon chung  wrote:
> 
> 
> 
> On 2018-01-11 01:48 AM, Thomas Bechtold wrote:
>> 
>> It was at lease for openSUSE: 
>> https://build.opensuse.org/package/show/Cloud:OpenStack:Pike/openstack-ceilometer
> 
> ah, maybe just centos then... or i'm not searching the correct place. :)
> 
> 
> -- 
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[gordon chung]

On 2018-01-11 01:48 AM, Thomas Bechtold wrote:
> 
> It was at lease for openSUSE: 
> https://build.opensuse.org/package/show/Cloud:OpenStack:Pike/openstack-ceilometer

ah, maybe just centos then... or i'm not searching the correct place. :)


-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Andreas Jaeger]

On 2018-01-11 07:48, Thomas Bechtold wrote:
> Hi,
> 
> On 11.01.2018 01:18, gordon chung wrote:
>>
>>
>> On 2018-01-10 06:44 PM, Doug Hellmann wrote:
 It's worth pointing out that openstacksdk has ceilometer REST API
 support in it, although it is special-cased since ceilometer was
 retired
 before we even made the service-types-authority:
>>
>> so ceilometer's REST API does not exist anymore. i don't believe it was
>> even packaged in Pike (at least i don't have have an rpm for it in my
>> environment).
> 
> It was at lease for openSUSE:
> https://build.opensuse.org/package/show/Cloud:OpenStack:Pike/openstack-ceilometer

Wrong package - statement still true:-)

https://build.opensuse.org/package/show/Cloud:OpenStack:Pike/python-ceilometerclient

Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Thomas Bechtold]

Hi,

On 11.01.2018 01:18, gordon chung wrote:



On 2018-01-10 06:44 PM, Doug Hellmann wrote:

It's worth pointing out that openstacksdk has ceilometer REST API
support in it, although it is special-cased since ceilometer was retired
before we even made the service-types-authority:


so ceilometer's REST API does not exist anymore. i don't believe it was
even packaged in Pike (at least i don't have have an rpm for it in my
environment).


It was at lease for openSUSE: 
https://build.opensuse.org/package/show/Cloud:OpenStack:Pike/openstack-ceilometer


Tom

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Emilien Macchi]

I'm favor of dropping Ceilometer API support in the SDK if we claim
1.0 will support Queens and beyond.
If the SDK has to support previous versions (Pike, Ocata, etc), then
we should warn the SDK users that Ceilometer API has been deprecated
and removed so depending on your cloud provider the SDK might not work
anymore.

Also, I'm in favor of supporting Gnocchi API in the SDK if that
something which makes sense for the Telemetry team.

On Wed, Jan 10, 2018 at 4:22 PM, Doug Hellmann  wrote:
> Excerpts from Monty Taylor's message of 2018-01-10 18:08:52 -0600:
>> On 01/10/2018 05:44 PM, Doug Hellmann wrote:
>> > Excerpts from Monty Taylor's message of 2018-01-10 17:40:28 -0600:
>> >> On 01/10/2018 04:10 PM, Jon Schlueter wrote:
>> >>> On Thu, Nov 23, 2017 at 4:12 PM, gordon chung  wrote:
>> 
>> 
>> 
>>  On 2017-11-22 04:18 AM, Julien Danjou wrote:
>> > Hi,
>> >
>> > Now that the Ceilometer API is gone, we really don't need
>> > ceilometerclient anymore. I've proposed a set of patches to retire it:
>> >
>> >  https://review.openstack.org/#/c/522183/
>> >
>> >>>
>> >>>
>> >>> So my question here is are we missing a process check for retiring a
>> >>> project that is still in
>> >>> the requirements of several other OpenStack projects?
>> >>>
>> >>> I went poking around and found that rally [4], heat [1], aodh [3] and
>> >>> mistral [2] still had references to
>> >>> ceilometerclient in the RPM packaging in RDO Queens, and on digging a
>> >>> bit more they
>> >>> were still in the requirements for at least those 4 projects.
>> >>>
>> >>> I would think that a discussion around retiring a project should also
>> >>> include at least enumerating
>> >>> which projects are currently consuming it [5].  That way a little bit
>> >>> of pressure on those consumers
>> >>> can be exerted to evaluate their usage of an about to be retired
>> >>> project.  It shouldn't stop the
>> >>> discussions around retiring a project just a data point for decision 
>> >>> making.
>> >>
>> >> It's worth pointing out that openstacksdk has ceilometer REST API
>> >> support in it, although it is special-cased since ceilometer was retired
>> >> before we even made the service-types-authority:
>> >>
>> >> http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/connection.py#n234
>>
>> Whoops, that's not ceilometer - that's gnocchi I think?
>>
>> ceilometer support *does* have a service-types-authority reference so
>> *isn't* special-cased and is here:
>>
>> http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/meter
>>
>> >> We can either keep it there indefinitely (there is no cost to keeping
>> >> it, other than that one "self._load('metric')" line) - or we could take
>> >> this opportunity to purge it from sdk as well.
>> >>
>> >> BUT - if we're going to remove it from SDK I'd rather we do it in the
>> >> very-near-future because we're getting closer to a 1.0 for SDK and once
>> >> that happens if ceilometer is still there ceilometer support will remain
>> >> until the end of recorded history.
>> >>
>> >> We could keep it and migrate the heat/mistral/rally/aodh
>> >> ceilometerclient uses to be SDK uses (although heaven knows how we test
>> >> that without a ceilometer in devstack)
>> >>
>> >> I honestly do not have a strong opinion in either direction and welcome
>> >> input on what people would like to see done.
>> >>
>> >> Monty
>> >>
>> >
>> > If ceilometer itself is deprecated, do we need to maintain support
>> > in any of our tools?
>>
>> We do not - although if we had had ceilometer support in shade I would
>> be very adamant that we continue to support it to the best of our
>> ability for forever, since you never know who out there is running on an
>> old cloud that still has it.
>>
>> This is why I could go either way personally from an SDK perspective -
>> we don't have a 1.0 release of SDK yet, so if we do think it's best to
>> just clean house, now's the time.
>>
>
> I favor dropping support in the SDK. I'm not sure what that means
> for the service projects that seem to be using it, though. Do they
> actually need it?
>
> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Doug Hellmann]

Excerpts from Monty Taylor's message of 2018-01-10 18:08:52 -0600:
> On 01/10/2018 05:44 PM, Doug Hellmann wrote:
> > Excerpts from Monty Taylor's message of 2018-01-10 17:40:28 -0600:
> >> On 01/10/2018 04:10 PM, Jon Schlueter wrote:
> >>> On Thu, Nov 23, 2017 at 4:12 PM, gordon chung  wrote:
> 
> 
> 
>  On 2017-11-22 04:18 AM, Julien Danjou wrote:
> > Hi,
> >
> > Now that the Ceilometer API is gone, we really don't need
> > ceilometerclient anymore. I've proposed a set of patches to retire it:
> >
> >  https://review.openstack.org/#/c/522183/
> >
> >>>
> >>>
> >>> So my question here is are we missing a process check for retiring a
> >>> project that is still in
> >>> the requirements of several other OpenStack projects?
> >>>
> >>> I went poking around and found that rally [4], heat [1], aodh [3] and
> >>> mistral [2] still had references to
> >>> ceilometerclient in the RPM packaging in RDO Queens, and on digging a
> >>> bit more they
> >>> were still in the requirements for at least those 4 projects.
> >>>
> >>> I would think that a discussion around retiring a project should also
> >>> include at least enumerating
> >>> which projects are currently consuming it [5].  That way a little bit
> >>> of pressure on those consumers
> >>> can be exerted to evaluate their usage of an about to be retired
> >>> project.  It shouldn't stop the
> >>> discussions around retiring a project just a data point for decision 
> >>> making.
> >>
> >> It's worth pointing out that openstacksdk has ceilometer REST API
> >> support in it, although it is special-cased since ceilometer was retired
> >> before we even made the service-types-authority:
> >>
> >> http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/connection.py#n234
> 
> Whoops, that's not ceilometer - that's gnocchi I think?
> 
> ceilometer support *does* have a service-types-authority reference so 
> *isn't* special-cased and is here:
> 
> http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/meter
> 
> >> We can either keep it there indefinitely (there is no cost to keeping
> >> it, other than that one "self._load('metric')" line) - or we could take
> >> this opportunity to purge it from sdk as well.
> >>
> >> BUT - if we're going to remove it from SDK I'd rather we do it in the
> >> very-near-future because we're getting closer to a 1.0 for SDK and once
> >> that happens if ceilometer is still there ceilometer support will remain
> >> until the end of recorded history.
> >>
> >> We could keep it and migrate the heat/mistral/rally/aodh
> >> ceilometerclient uses to be SDK uses (although heaven knows how we test
> >> that without a ceilometer in devstack)
> >>
> >> I honestly do not have a strong opinion in either direction and welcome
> >> input on what people would like to see done.
> >>
> >> Monty
> >>
> > 
> > If ceilometer itself is deprecated, do we need to maintain support
> > in any of our tools?
> 
> We do not - although if we had had ceilometer support in shade I would 
> be very adamant that we continue to support it to the best of our 
> ability for forever, since you never know who out there is running on an 
> old cloud that still has it.
> 
> This is why I could go either way personally from an SDK perspective - 
> we don't have a 1.0 release of SDK yet, so if we do think it's best to 
> just clean house, now's the time.
> 

I favor dropping support in the SDK. I'm not sure what that means
for the service projects that seem to be using it, though. Do they
actually need it?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[gordon chung]

On 2018-01-10 06:44 PM, Doug Hellmann wrote:
>> It's worth pointing out that openstacksdk has ceilometer REST API
>> support in it, although it is special-cased since ceilometer was retired
>> before we even made the service-types-authority:

so ceilometer's REST API does not exist anymore. i don't believe it was 
even packaged in Pike (at least i don't have have an rpm for it in my 
environment).

>>
>> http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/connection.py#n234
>>
>> We can either keep it there indefinitely (there is no cost to keeping
>> it, other than that one "self._load('metric')" line) - or we could take
>> this opportunity to purge it from sdk as well.
>>
>> BUT - if we're going to remove it from SDK I'd rather we do it in the
>> very-near-future because we're getting closer to a 1.0 for SDK and once
>> that happens if ceilometer is still there ceilometer support will remain
>> until the end of recorded history.

if it was removed from SDK, does it affect installations from pre-Pike? 
technically the API code exists prior to Pike (but we've been telling 
people for a year+ prior to that, to stop using it). if it only affects 
Queens onwards, i'm an easy yes to removing for openstacksdk 1.0.

>>
>> We could keep it and migrate the heat/mistral/rally/aodh
>> ceilometerclient uses to be SDK uses (although heaven knows how we test
>> that without a ceilometer in devstack)
>>
i'm guessing it's not tested anywhere as we've removed the API code for 
a few months now and have not heard anyone complain about a broken gate.

> If ceilometer itself is deprecated, do we need to maintain support
> in any of our tools?

just to clarify, ceilometer itself is **not** deprecated. it just 
doesn't have an API as there is currently nothing to query/interact with 
remotely. jd had an idea how to manage/monitor existing agents but that 
is unrealised currently.

the workflow remains as it has been:
- ceilometer agents generate/normalise data relating to openstack resources
- ceilometer data is pushed to a configurable target for consumption
- gnocchi, panko, whatever you want
- you interact with the data according to the specific targets.

cheers,
-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Monty Taylor]

On 01/10/2018 05:44 PM, Doug Hellmann wrote:

Excerpts from Monty Taylor's message of 2018-01-10 17:40:28 -0600:

On 01/10/2018 04:10 PM, Jon Schlueter wrote:

On Thu, Nov 23, 2017 at 4:12 PM, gordon chung  wrote:




On 2017-11-22 04:18 AM, Julien Danjou wrote:

Hi,

Now that the Ceilometer API is gone, we really don't need
ceilometerclient anymore. I've proposed a set of patches to retire it:

 https://review.openstack.org/#/c/522183/




So my question here is are we missing a process check for retiring a
project that is still in
the requirements of several other OpenStack projects?

I went poking around and found that rally [4], heat [1], aodh [3] and
mistral [2] still had references to
ceilometerclient in the RPM packaging in RDO Queens, and on digging a
bit more they
were still in the requirements for at least those 4 projects.

I would think that a discussion around retiring a project should also
include at least enumerating
which projects are currently consuming it [5].  That way a little bit
of pressure on those consumers
can be exerted to evaluate their usage of an about to be retired
project.  It shouldn't stop the
discussions around retiring a project just a data point for decision making.


It's worth pointing out that openstacksdk has ceilometer REST API
support in it, although it is special-cased since ceilometer was retired
before we even made the service-types-authority:

http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/connection.py#n234


Whoops, that's not ceilometer - that's gnocchi I think?

ceilometer support *does* have a service-types-authority reference so 
*isn't* special-cased and is here:


http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/meter


We can either keep it there indefinitely (there is no cost to keeping
it, other than that one "self._load('metric')" line) - or we could take
this opportunity to purge it from sdk as well.

BUT - if we're going to remove it from SDK I'd rather we do it in the
very-near-future because we're getting closer to a 1.0 for SDK and once
that happens if ceilometer is still there ceilometer support will remain
until the end of recorded history.

We could keep it and migrate the heat/mistral/rally/aodh
ceilometerclient uses to be SDK uses (although heaven knows how we test
that without a ceilometer in devstack)

I honestly do not have a strong opinion in either direction and welcome
input on what people would like to see done.

Monty



If ceilometer itself is deprecated, do we need to maintain support
in any of our tools?


We do not - although if we had had ceilometer support in shade I would 
be very adamant that we continue to support it to the best of our 
ability for forever, since you never know who out there is running on an 
old cloud that still has it.


This is why I could go either way personally from an SDK perspective - 
we don't have a 1.0 release of SDK yet, so if we do think it's best to 
just clean house, now's the time.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Doug Hellmann]

Excerpts from Monty Taylor's message of 2018-01-10 17:40:28 -0600:
> On 01/10/2018 04:10 PM, Jon Schlueter wrote:
> > On Thu, Nov 23, 2017 at 4:12 PM, gordon chung  wrote:
> >>
> >>
> >>
> >> On 2017-11-22 04:18 AM, Julien Danjou wrote:
> >>> Hi,
> >>>
> >>> Now that the Ceilometer API is gone, we really don't need
> >>> ceilometerclient anymore. I've proposed a set of patches to retire it:
> >>>
> >>> https://review.openstack.org/#/c/522183/
> >>>
> > 
> > 
> > So my question here is are we missing a process check for retiring a
> > project that is still in
> > the requirements of several other OpenStack projects?
> > 
> > I went poking around and found that rally [4], heat [1], aodh [3] and
> > mistral [2] still had references to
> > ceilometerclient in the RPM packaging in RDO Queens, and on digging a
> > bit more they
> > were still in the requirements for at least those 4 projects.
> > 
> > I would think that a discussion around retiring a project should also
> > include at least enumerating
> > which projects are currently consuming it [5].  That way a little bit
> > of pressure on those consumers
> > can be exerted to evaluate their usage of an about to be retired
> > project.  It shouldn't stop the
> > discussions around retiring a project just a data point for decision making.
> 
> It's worth pointing out that openstacksdk has ceilometer REST API 
> support in it, although it is special-cased since ceilometer was retired 
> before we even made the service-types-authority:
> 
> http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/connection.py#n234
> 
> We can either keep it there indefinitely (there is no cost to keeping 
> it, other than that one "self._load('metric')" line) - or we could take 
> this opportunity to purge it from sdk as well.
> 
> BUT - if we're going to remove it from SDK I'd rather we do it in the 
> very-near-future because we're getting closer to a 1.0 for SDK and once 
> that happens if ceilometer is still there ceilometer support will remain 
> until the end of recorded history.
> 
> We could keep it and migrate the heat/mistral/rally/aodh 
> ceilometerclient uses to be SDK uses (although heaven knows how we test 
> that without a ceilometer in devstack)
> 
> I honestly do not have a strong opinion in either direction and welcome 
> input on what people would like to see done.
> 
> Monty
> 

If ceilometer itself is deprecated, do we need to maintain support
in any of our tools?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Monty Taylor]

On 01/10/2018 04:10 PM, Jon Schlueter wrote:

On Thu, Nov 23, 2017 at 4:12 PM, gordon chung  wrote:




On 2017-11-22 04:18 AM, Julien Danjou wrote:

Hi,

Now that the Ceilometer API is gone, we really don't need
ceilometerclient anymore. I've proposed a set of patches to retire it:

https://review.openstack.org/#/c/522183/




So my question here is are we missing a process check for retiring a
project that is still in
the requirements of several other OpenStack projects?

I went poking around and found that rally [4], heat [1], aodh [3] and
mistral [2] still had references to
ceilometerclient in the RPM packaging in RDO Queens, and on digging a
bit more they
were still in the requirements for at least those 4 projects.

I would think that a discussion around retiring a project should also
include at least enumerating
which projects are currently consuming it [5].  That way a little bit
of pressure on those consumers
can be exerted to evaluate their usage of an about to be retired
project.  It shouldn't stop the
discussions around retiring a project just a data point for decision making.


It's worth pointing out that openstacksdk has ceilometer REST API 
support in it, although it is special-cased since ceilometer was retired 
before we even made the service-types-authority:


http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/connection.py#n234

We can either keep it there indefinitely (there is no cost to keeping 
it, other than that one "self._load('metric')" line) - or we could take 
this opportunity to purge it from sdk as well.


BUT - if we're going to remove it from SDK I'd rather we do it in the 
very-near-future because we're getting closer to a 1.0 for SDK and once 
that happens if ceilometer is still there ceilometer support will remain 
until the end of recorded history.


We could keep it and migrate the heat/mistral/rally/aodh 
ceilometerclient uses to be SDK uses (although heaven knows how we test 
that without a ceilometer in devstack)


I honestly do not have a strong opinion in either direction and welcome 
input on what people would like to see done.


Monty

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Doug Hellmann]

Excerpts from gordon chung's message of 2018-01-10 23:28:05 +:
> 
> On 2018-01-10 05:10 PM, Jon Schlueter wrote:
> > I would think that a discussion around retiring a project should also
> > include at least enumerating
> > which projects are currently consuming it [5].  That way a little bit
> > of pressure on those consumers
> > can be exerted to evaluate their usage of an about to be retired
> > project.  It shouldn't stop the
> > discussions around retiring a project just a data point for decision making.
> 
> this is a very valid point. this is something overlooked on my part.
> 
> out of curiosity, but what's the effect have 'retiring' something in 
> openstack and having services still referencing ceilometerclient? is it 
> that it will not get packaged in centos/ubuntu and therefore will be 
> missing requirements when installed? or can you not build the package at 
> all?
> 
> cheers,
> 

The python-ceilometer client is empty now except for a README file
explaining that the project is retired. So if there's a bug in the
library, there's no convenient way for anyone to fix it.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[gordon chung]

On 2018-01-10 05:10 PM, Jon Schlueter wrote:
> I would think that a discussion around retiring a project should also
> include at least enumerating
> which projects are currently consuming it [5].  That way a little bit
> of pressure on those consumers
> can be exerted to evaluate their usage of an about to be retired
> project.  It shouldn't stop the
> discussions around retiring a project just a data point for decision making.

this is a very valid point. this is something overlooked on my part.

out of curiosity, but what's the effect have 'retiring' something in 
openstack and having services still referencing ceilometerclient? is it 
that it will not get packaged in centos/ubuntu and therefore will be 
missing requirements when installed? or can you not build the package at 
all?

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[Jon Schlueter]

On Thu, Nov 23, 2017 at 4:12 PM, gordon chung  wrote:
>
>
>
> On 2017-11-22 04:18 AM, Julien Danjou wrote:
> > Hi,
> >
> > Now that the Ceilometer API is gone, we really don't need
> > ceilometerclient anymore. I've proposed a set of patches to retire it:
> >
> >https://review.openstack.org/#/c/522183/
> >


So my question here is are we missing a process check for retiring a
project that is still in
the requirements of several other OpenStack projects?

I went poking around and found that rally [4], heat [1], aodh [3] and
mistral [2] still had references to
ceilometerclient in the RPM packaging in RDO Queens, and on digging a
bit more they
were still in the requirements for at least those 4 projects.

I would think that a discussion around retiring a project should also
include at least enumerating
which projects are currently consuming it [5].  That way a little bit
of pressure on those consumers
can be exerted to evaluate their usage of an about to be retired
project.  It shouldn't stop the
discussions around retiring a project just a data point for decision making.

Thanks

Jon Schlueter


[1] https://review.openstack.org/532617 - heat
[2] https://review.openstack.org/532610 - mistral
[3] https://review.openstack.org/526246 - aodh
[4] https://github.com/openstack/rally/blob/master/requirements.txt#L34
[5] 
http://codesearch.openstack.org/?q=python-ceilometerclient=nope=requirements.txt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the cpu util

[Jaze Lee]

2017-12-20 23:48 GMT+08:00 gordon chung :
>
>
> On 2017-12-20 10:06 AM, Jaze Lee wrote:
>> Yes, i notice it. But when there will be vcpu.*.time? Which libvirt version?
>> We use libvirt 3.2.0 and it does not have vcpu.*.time, so it will go
>> to fall back.
>
> i have:
>
> virsh # version
> Compiled against library: libvirt 3.2.0
> Using library: libvirt 3.2.0
> Using API: QEMU 3.2.0
> Running hypervisor: QEMU 2.9.0
>
> and it works fine. possibly related to your qemu version?
No. the same as yours

>
>>
>> The fall back is so coarse, do you think we should have it?
>
> are you overcommiting your cpu? that is exactly the case that breaks
> down as the patch i sent you mentions. please look at the bugs and
> related threads on that patch.
I found the root cause. Becuase we use quickstart in libivrt. The vm running
on host is ok. But the vm running in libvirt vm can not get vcpu.x.time.

Thanks gord.


>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the cpu util

[gordon chung]

On 2017-12-20 10:06 AM, Jaze Lee wrote:
> Yes, i notice it. But when there will be vcpu.*.time? Which libvirt version?
> We use libvirt 3.2.0 and it does not have vcpu.*.time, so it will go
> to fall back.

i have:

virsh # version
Compiled against library: libvirt 3.2.0
Using library: libvirt 3.2.0
Using API: QEMU 3.2.0
Running hypervisor: QEMU 2.9.0

and it works fine. possibly related to your qemu version?

> 
> The fall back is so coarse, do you think we should have it?

are you overcommiting your cpu? that is exactly the case that breaks 
down as the patch i sent you mentions. please look at the bugs and 
related threads on that patch.

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the cpu util

[Jaze Lee]

2017-12-20 22:29 GMT+08:00 gordon chung :
>
>
> On 2017-12-20 03:14 AM, Jaze Lee wrote:
>> Hello,
>> We use ceilometer newton, and also master calculates cpu same as newton.
>> It is in 
>> https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/virt/libvirt/inspector.py#L183
>>
>> I test with a two vcpu vm, and find using this method as line 183
>> we can not get cpu util 100, always 50 or 52.
>>
>> The test script is here(run on compute node, and there is only one vm)
>> #!/bin/bash
>>
>> cpu_time_begin=`virsh  domstats --cpu-total | grep cpu.time| awk -F
>> '=' '{print $2}'`
>> sleep 1
>> cpu_time_end=`virsh  domstats --cpu-total | grep cpu.time| awk -F '='
>> '{print $2}'`
>>
>> cpu_util=$(((cpu_time_end - cpu_time_begin) * 100 / 10**9))
>> echo "cpu util is ", $cpu_util
>>
>> The output will be 100 or 101, if we divide cpu_util with core number,
>> then we definitely get 50. But in the vm we watch with top and find
>> all core is 99%.
>>
>> Can someone tell me why we only use cpu_time here? May be we should
>> add cpu.user and cpu.system? It is insane only get 50% when the vm is
>> totally 100%
>>
>
> does adding cpu.user and cpu.system make sense or are you just adding
> random numbers in hopes it gets you to right number? :P
The latter one. :)


>
> i'm not a libvirt/qemu dev but it seems cpu.system is already part of
> cpu.time[1]
>
> if you look further into the code, you should see that the agent tries
> to first use vcpu.*.time to build more accurate cpu.time value. if you
> look at patch[3], it should give you more information as to why and what
> requirements you need.

Yes, i notice it. But when there will be vcpu.*.time? Which libvirt version?
We use libvirt 3.2.0 and it does not have vcpu.*.time, so it will go
to fall back.

The fall back is so coarse, do you think we should have it?


>
>
> [1]
> https://stackoverflow.com/questions/40468370/what-does-cpu-time-represent-exactly-in-libvirt
> [2]
> https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/virt/libvirt/inspector.py#L175-L176
> [3]
> https://github.com/openstack/ceilometer/commit/a4ec0911a3ed4137a1c832fbd7c8fee80c7d4601
>
> cheers,
>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the cpu util

[gordon chung]

On 2017-12-20 03:14 AM, Jaze Lee wrote:
> Hello,
> We use ceilometer newton, and also master calculates cpu same as newton.
> It is in 
> https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/virt/libvirt/inspector.py#L183
> 
> I test with a two vcpu vm, and find using this method as line 183
> we can not get cpu util 100, always 50 or 52.
> 
> The test script is here(run on compute node, and there is only one vm)
> #!/bin/bash
> 
> cpu_time_begin=`virsh  domstats --cpu-total | grep cpu.time| awk -F
> '=' '{print $2}'`
> sleep 1
> cpu_time_end=`virsh  domstats --cpu-total | grep cpu.time| awk -F '='
> '{print $2}'`
> 
> cpu_util=$(((cpu_time_end - cpu_time_begin) * 100 / 10**9))
> echo "cpu util is ", $cpu_util
> 
> The output will be 100 or 101, if we divide cpu_util with core number,
> then we definitely get 50. But in the vm we watch with top and find
> all core is 99%.
> 
> Can someone tell me why we only use cpu_time here? May be we should
> add cpu.user and cpu.system? It is insane only get 50% when the vm is
> totally 100%
> 

does adding cpu.user and cpu.system make sense or are you just adding 
random numbers in hopes it gets you to right number? :P

i'm not a libvirt/qemu dev but it seems cpu.system is already part of 
cpu.time[1]

if you look further into the code, you should see that the agent tries 
to first use vcpu.*.time to build more accurate cpu.time value. if you 
look at patch[3], it should give you more information as to why and what 
requirements you need.


[1] 
https://stackoverflow.com/questions/40468370/what-does-cpu-time-represent-exactly-in-libvirt
[2] 
https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/virt/libvirt/inspector.py#L175-L176
[3] 
https://github.com/openstack/ceilometer/commit/a4ec0911a3ed4137a1c832fbd7c8fee80c7d4601

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] the cpu util

[Jaze Lee]

Hello,
   We use ceilometer newton, and also master calculates cpu same as newton.
   It is in 
https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/virt/libvirt/inspector.py#L183

   I test with a two vcpu vm, and find using this method as line 183
we can not get cpu util 100, always 50 or 52.

The test script is here(run on compute node, and there is only one vm)
#!/bin/bash

cpu_time_begin=`virsh  domstats --cpu-total | grep cpu.time| awk -F
'=' '{print $2}'`
sleep 1
cpu_time_end=`virsh  domstats --cpu-total | grep cpu.time| awk -F '='
'{print $2}'`

cpu_util=$(((cpu_time_end - cpu_time_begin) * 100 / 10**9))
echo "cpu util is ", $cpu_util

The output will be 100 or 101, if we divide cpu_util with core number,
then we definitely get 50. But in the vm we watch with top and find
all core is 99%.

Can someone tell me why we only use cpu_time here? May be we should
add cpu.user and cpu.system? It is insane only get 50% when the vm is
totally 100%

Thanks a lot guys...




-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[gordon chung]

On 2017-12-18 10:28 AM, Jaze Lee wrote:
> Is there some document about how to use rate? I do not find any about it
> Thanks.
> 

https://gnocchi.xyz/rest.html#archive-policy

its' similar to how other aggregates are defined. it's just not part of 
default aggregation methods.

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[Jaze Lee]

2017-12-18 21:17 GMT+08:00 gordon chung :
>
>
> On 2017-12-17 11:08 PM, Jaze Lee wrote:
>> Hello.
>>
>> 2017-12-16 9:30 GMT+08:00 Jaze Lee :
>>> 2017-12-15 23:17 GMT+08:00 gordon chung :


 On 2017-12-14 10:38 PM, Jaze Lee wrote:
> It sounds like great. When the gnocchi can be ready to do transformer's 
> work?
> If it takes long time, we have to move to compute temporarily.

 it already exists in gnocchi 4.1+. we just need to change the workflow
 in ceilometer so it integrates with gnocchi.
>>> Oh, I do not quite understand, can you give more details?
>>> I do not know change which part of workflow in ceilometer.
>>>
>
> i don't openstack on weekends.
Sorry...

>
> what needs to change is that rate aggregate needs to be enabled for
> specific meters such as cpu. this will effectively calculate the cputime
> per second. then you apply the same maths on timeseries that transformer
> does when you query gnocchi.

Is there some document about how to use rate? I do not find any about it
Thanks.

>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[gordon chung]

On 2017-12-17 11:08 PM, Jaze Lee wrote:
> Hello.
> 
> 2017-12-16 9:30 GMT+08:00 Jaze Lee :
>> 2017-12-15 23:17 GMT+08:00 gordon chung :
>>>
>>>
>>> On 2017-12-14 10:38 PM, Jaze Lee wrote:
 It sounds like great. When the gnocchi can be ready to do transformer's 
 work?
 If it takes long time, we have to move to compute temporarily.
>>>
>>> it already exists in gnocchi 4.1+. we just need to change the workflow
>>> in ceilometer so it integrates with gnocchi.
>> Oh, I do not quite understand, can you give more details?
>> I do not know change which part of workflow in ceilometer.
>>

i don't openstack on weekends.

what needs to change is that rate aggregate needs to be enabled for 
specific meters such as cpu. this will effectively calculate the cputime 
per second. then you apply the same maths on timeseries that transformer 
does when you query gnocchi.

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[Jaze Lee]

Hello.

2017-12-16 9:30 GMT+08:00 Jaze Lee :
> 2017-12-15 23:17 GMT+08:00 gordon chung :
>>
>>
>> On 2017-12-14 10:38 PM, Jaze Lee wrote:
>>> It sounds like great. When the gnocchi can be ready to do transformer's 
>>> work?
>>> If it takes long time, we have to move to compute temporarily.
>>
>> it already exists in gnocchi 4.1+. we just need to change the workflow
>> in ceilometer so it integrates with gnocchi.
> Oh, I do not quite understand, can you give more details?
> I do not know change which part of workflow in ceilometer.
>
>
>>
>>
>> --
>> gord
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> 谦谦君子



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[Jaze Lee]

2017-12-15 23:17 GMT+08:00 gordon chung :
>
>
> On 2017-12-14 10:38 PM, Jaze Lee wrote:
>> It sounds like great. When the gnocchi can be ready to do transformer's work?
>> If it takes long time, we have to move to compute temporarily.
>
> it already exists in gnocchi 4.1+. we just need to change the workflow
> in ceilometer so it integrates with gnocchi.
Oh, I do not quite understand, can you give more details?
I do not know change which part of workflow in ceilometer.


>
>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[gordon chung]

On 2017-12-14 10:38 PM, Jaze Lee wrote:
> It sounds like great. When the gnocchi can be ready to do transformer's work?
> If it takes long time, we have to move to compute temporarily.

it already exists in gnocchi 4.1+. we just need to change the workflow 
in ceilometer so it integrates with gnocchi.


-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[Jaze Lee]

2017-12-14 22:34 GMT+08:00 gordon chung :
>
>
> On 2017-12-14 04:14 AM, Jaze Lee wrote:
>>> The plan is to actually make the TSDB (Gnocchi) do that job for us.
>> What ？
>> Is there some detail on this plan ?  If it did, then we do not need
>> workload partition, and scale with agent process happily
>>
>
> Gnocchi can capture rate information and can also have mathematical
> operations applied to it. there are some examples here:
> https://gnocchi.xyz/rest.html#examples. i imagine we'll change the gate
> workflow to that eventually.
It sounds like great. When the gnocchi can be ready to do transformer's work?
If it takes long time, we have to move to compute temporarily.

>
> you will still need partitioning for HA of central agents... and
> currently, the only way to get batch support is with workload_partitioning.
>
> but yes, moving the transformation to storage will provide the most
> durable solution for transformations.
>
> cheers,
>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[gordon chung]

On 2017-12-14 04:14 AM, Jaze Lee wrote:
>> The plan is to actually make the TSDB (Gnocchi) do that job for us.
> What ？
> Is there some detail on this plan ?  If it did, then we do not need
> workload partition, and scale with agent process happily
> 

Gnocchi can capture rate information and can also have mathematical 
operations applied to it. there are some examples here: 
https://gnocchi.xyz/rest.html#examples. i imagine we'll change the gate 
workflow to that eventually.

you will still need partitioning for HA of central agents... and 
currently, the only way to get batch support is with workload_partitioning.

but yes, moving the transformation to storage will provide the most 
durable solution for transformations.

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[Jaze Lee]

2017-12-14 16:58 GMT+08:00 Julien Danjou :
> On Thu, Dec 14 2017, Jaze Lee wrote:
>
>> Oh, sorry.  I found libvirt can not get cpu.util, cpu.delta, and
>> network/disk rate info directly
>> from inspector. So i mean, we can move this into compute pollster.
>> Before it sends to notifications.sample,
>> transform it. Then we can scale notification agent and do not have
>> flapping sample problems
>
> The plan is to actually make the TSDB (Gnocchi) do that job for us.
What ？
Is there some detail on this plan ?  If it did, then we do not need
workload partition, and scale with agent process happily


>
> --
> Julien Danjou
> /* Free Software hacker
>https://julien.danjou.info */



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[Julien Danjou]

On Thu, Dec 14 2017, Jaze Lee wrote:

> Oh, sorry.  I found libvirt can not get cpu.util, cpu.delta, and
> network/disk rate info directly
> from inspector. So i mean, we can move this into compute pollster.
> Before it sends to notifications.sample,
> transform it. Then we can scale notification agent and do not have
> flapping sample problems

The plan is to actually make the TSDB (Gnocchi) do that job for us.

-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[Jaze Lee]

2017-12-13 21:16 GMT+08:00 gordon chung :
>
>
> On 2017-12-13 01:52 AM, Jaze Lee wrote:
>> Hello,
>>
>>  What about move transformer to compute pollster? If we do this, we
>> do not need
>>  transformer any more
>
> sorry, i don't understand. if you move the transformer, you still have a
> transformer.

Oh, sorry.  I found libvirt can not get cpu.util, cpu.delta, and
network/disk rate info directly
from inspector. So i mean, we can move this into compute pollster.
Before it sends to notifications.sample,
transform it. Then we can scale notification agent and do not have
flapping sample problems


>
>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about add transformer into libvirt

[gordon chung]

On 2017-12-13 01:52 AM, Jaze Lee wrote:
> Hello,
> 
>  What about move transformer to compute pollster? If we do this, we
> do not need
>  transformer any more

sorry, i don't understand. if you move the transformer, you still have a 
transformer.


-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] about add transformer into libvirt

[Jaze Lee]

Hello,

What about move transformer to compute pollster? If we do this, we
do not need
transformer any more



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][gnocchi] some code not very understood

[Julien Danjou]

On Thu, Dec 07 2017, Jaze Lee wrote:

> Yes. I misunderstand here. If there any exception in gnocchi api, it
> won't store any measures.
> Just after all resource, and all metrics are there, gnocchi will store
> those measures. Am i right?

Yes.

-- 
Julien Danjou
# Free Software hacker
# https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][gnocchi] some code not very understood

[Jaze Lee]

2017-12-07 16:36 GMT+08:00 Julien Danjou :
> On Thu, Dec 07 2017, Jaze Lee wrote:
>
>>I think at line 416, we should remove some measures that already post at 
>> 390.
>>Or may be i miss something?
>>
>>
>> https://github.com/openstack/ceilometer/blob/master/ceilometer/publisher/gnocchi.py#L416
>
> The code at L416 is only executed if L390 raises an exception.
> So if L390 raises an exception, it has not complete. That's why it's
> done again here.
Yes. I misunderstand here. If there any exception in gnocchi api, it
won't store any measures.
Just after all resource, and all metrics are there, gnocchi will store
those measures. Am i right?



>
> --
> Julien Danjou
> # Free Software hacker
> # https://julien.danjou.info



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][gnocchi] some code not very understood

[Julien Danjou]

On Thu, Dec 07 2017, Jaze Lee wrote:

>I think at line 416, we should remove some measures that already post at 
> 390.
>Or may be i miss something?
>
>
> https://github.com/openstack/ceilometer/blob/master/ceilometer/publisher/gnocchi.py#L416

The code at L416 is only executed if L390 raises an exception.
So if L390 raises an exception, it has not complete. That's why it's
done again here.

-- 
Julien Danjou
# Free Software hacker
# https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][gnocchi] some code not very understood

[Jaze Lee]

Hello,

   I think at line 416, we should remove some measures that already post at 390.
   Or may be i miss something?

   
https://github.com/openstack/ceilometer/blob/master/ceilometer/publisher/gnocchi.py#L416





-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[Julien Danjou]

On Mon, Dec 04 2017, gordon chung wrote:

> but at the end of the day, are you building for reality or for some 
> personal ideal scenario? :p

Haha, you know that if I was building for an ideal scenario, most of
Ceilometer would probably have never existed in this way.

Half of Ceilometer is already hacking around OpenStack project
limitation for the last 5 years.

> is there a service that offers a stats endpoint?

Outside of OpenStack? Plenty. :)

-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[gordon chung]

On 2017-12-04 08:53 AM, Julien Danjou wrote:
> Which is usually a problem because of the APIs, not Ceilometer. They're
> sometimes utterly slow for simple things and other times don't offer a
> way to retrieve what Ceilometer needs in a batch-y way.
> 
>> in general, there's a preference that stats be pushed to ceilometer
>> rather than pulled.
> Yes, but for the wrong reasons. Having a component sending a batch of
> notification every hour without any configuration knob and granularity
> choice is a PITA for the users. Plus it ususally comes from a single
> point (of failure?).

well you can configure it, just at a per service level.. but that's a 
fair point.

> 
> Ceilometer would do a better job than $project at getting this info, if
> said project(s) would offer a proper and performant API to retrieve them
> efficiently.

but at the end of the day, are you building for reality or for some 
personal ideal scenario? :p is there a service that offers a stats endpoint?

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[Duncan Thomas]

Either way step one is to propose the code for a better replacement. I
doubt you'll get much opposition to that.

No need to propose removing the old code immediately, it can live in
parallel for as long as needed.

On 4 December 2017 at 15:14, Jaze Lee  wrote:
> 2017-12-04 19:36 GMT+08:00 Duncan Thomas :
>> Why remove something that works and people are using?
>
> Yes, removing it will make noise.
>
>>
>> If polster can be set up to do the job, then great, but there's no
>> rush to remove the existing infrastructure; this is one case where
>> duplication is very, very cheap.
>
> Yes it is too cheap to refuse. But i think it should be not exists there.
>
>>
>> On 4 December 2017 at 10:30, Jaze Lee  wrote:
>>> Hello,
>>>
>>> Right now,we can get volume from central pollester. Then i think
>>> may be we can drop cinder volume usage.
>>>Cinder volume usage, do not like nova usage which is in nova
>>> compute. It is a cmd.  And should put it in cron. This will cause more
>>> work in devops. If ceilometer
>>> pollester can handle it. I think may be we should remove it?
>>>
>>>
>>>
>>>
>>>
>>> --
>>> 谦谦君子
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> --
>> Duncan Thomas
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> 谦谦君子
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Duncan Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] remove gnocchi http interface?

[gordon chung]

On 2017-12-04 09:52 AM, Jaze Lee wrote:
> 2017-12-04 20:53 GMT+08:00 gordon chung :
>> i'm curious. how does a "simple tcp socket" protect ou against
> 
> I do not quite understand 'protect ou against'.
> The tcp socket means it is a client to send samples to gnocchi tcp server.
> 

oh sorry, i meant "protect you against". i'm just wondering how this tcp 
server authenticates? and if not, why not just have two APIs where one 
is user facing and authenticates, and internally another that doesn't 
for ceilometer->gnocchi?

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[Jaze Lee]

2017-12-04 19:36 GMT+08:00 Duncan Thomas :
> Why remove something that works and people are using?

Yes, removing it will make noise.

>
> If polster can be set up to do the job, then great, but there's no
> rush to remove the existing infrastructure; this is one case where
> duplication is very, very cheap.

Yes it is too cheap to refuse. But i think it should be not exists there.

>
> On 4 December 2017 at 10:30, Jaze Lee  wrote:
>> Hello,
>>
>> Right now,we can get volume from central pollester. Then i think
>> may be we can drop cinder volume usage.
>>Cinder volume usage, do not like nova usage which is in nova
>> compute. It is a cmd.  And should put it in cron. This will cause more
>> work in devops. If ceilometer
>> pollester can handle it. I think may be we should remove it?
>>
>>
>>
>>
>>
>> --
>> 谦谦君子
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Duncan Thomas
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[Jaze Lee]

2017-12-04 20:57 GMT+08:00 gordon chung :
>
>
> On 2017-12-04 05:30 AM, Jaze Lee wrote:
>>  Right now,we can get volume from central pollester. Then i think
>> may be we can drop cinder volume usage.
>
> which metrics are you referring to? just for the record, except for
> libvirt stats, the polling done by ceilometer is against the API which
> may create additional load.

I mean volume.exists snapshot.exists, and backup.exists

>
> in general, there's a preference that stats be pushed to ceilometer
> rather than pulled.

The cinder volume usage cmd is too difficult to use in production. I
do not know
why don't learn from nova?  Put it in a periodic task, and can be configurable.


>
> cheers,
>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] remove gnocchi http interface?

[Jaze Lee]

2017-12-04 20:53 GMT+08:00 gordon chung :
>
>
> On 2017-12-03 10:41 PM, Jaze Lee wrote:
>> So what about to remove gnocchi http, and add a simple tcp socket, which
>>   will send to gnocchi tcp server(which will also be created.).
>
> i'm curious. how does a "simple tcp socket" protect ou against

I do not quite understand 'protect ou against'.
The tcp socket means it is a client to send samples to gnocchi tcp server.


>
>  > anyone can update gnocchi database which is not we want
>
> --
> gord
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[Julien Danjou]

On Mon, Dec 04 2017, gordon chung wrote:

> On 2017-12-04 05:30 AM, Jaze Lee wrote:
>>  Right now,we can get volume from central pollester. Then i think
>> may be we can drop cinder volume usage.
>
> which metrics are you referring to? just for the record, except for 
> libvirt stats, the polling done by ceilometer is against the API which 
> may create additional load.

Which is usually a problem because of the APIs, not Ceilometer. They're
sometimes utterly slow for simple things and other times don't offer a
way to retrieve what Ceilometer needs in a batch-y way.

> in general, there's a preference that stats be pushed to ceilometer 
> rather than pulled.

Yes, but for the wrong reasons. Having a component sending a batch of
notification every hour without any configuration knob and granularity
choice is a PITA for the users. Plus it ususally comes from a single
point (of failure?).

Ceilometer would do a better job than $project at getting this info, if
said project(s) would offer a proper and performant API to retrieve them
efficiently.

-- 
Julien Danjou
-- Free Software hacker
-- https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about workload partition

[gordon chung]

On 2017-12-03 10:30 PM, 李田清 wrote:
> 
> 
> On 2017-12-01 05:03 AM, 李田清 wrote:
>  >> Hello,
>  >>       we test workload partition, and find it's much slower than not
>  >> using it.
>  >>       After some review, we find that, after get samples from
>  >> notifications.sample
>  >>       ceilometer unpacks them and sends them one by one to the pipe
>  >>       ceilometer.pipe.*, this will make the consumer slow. Right now,
>  >> the rabbit_qos_prefetch_count to 1. If we sent it to 10, the connection
>  >> will be reset
> 
>  > currently, i believe rabbit_qos_prefetch_count will be set to whatever
>  > value you set batch_size to.
> You mean in the past, it can not be set to whatever?
> We test newton, and find under 1k vm, if we open workload partition,
> it will be reset regularly.
> 
>  >i'll give a two part answer but first i'll start with a question: what
>  >version of oslo.messaging do you have?
> 
> newton 5.10.2
> 

i just checked, oslo.messaging==5.10.2 has the offending patch that 
decreases performance significantly. as newton is EOL, it seems you have 
a few choices:
- revert to 5.10.1 but i'm not sure if that reintroduces old bugs
- manually patch your oslo.messaging with: 
https://review.openstack.org/#/c/524099
- pull in a newer oslo.messaging from another branch (fix is unreleased 
currently)
- manually patch notification agent to use multiple threads[1] (but this 
will possibly add some instability to your transforms)
- disable workload_partitioning

option 2 or 5 are probably the safest choices depending on your load.

[1] 
https://github.com/openstack/ceilometer/blob/newton-eol/ceilometer/notification.py#L305-L307

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[gordon chung]

On 2017-12-04 05:30 AM, Jaze Lee wrote:
>  Right now,we can get volume from central pollester. Then i think
> may be we can drop cinder volume usage.

which metrics are you referring to? just for the record, except for 
libvirt stats, the polling done by ceilometer is against the API which 
may create additional load.

in general, there's a preference that stats be pushed to ceilometer 
rather than pulled.

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] remove gnocchi http interface?

[gordon chung]

On 2017-12-03 10:41 PM, Jaze Lee wrote:
> So what about to remove gnocchi http, and add a simple tcp socket, which
>   will send to gnocchi tcp server(which will also be created.).

i'm curious. how does a "simple tcp socket" protect ou against

 > anyone can update gnocchi database which is not we want

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][cinder]about cinder volume usage

[Duncan Thomas]

Why remove something that works and people are using?

If polster can be set up to do the job, then great, but there's no
rush to remove the existing infrastructure; this is one case where
duplication is very, very cheap.

On 4 December 2017 at 10:30, Jaze Lee  wrote:
> Hello,
>
> Right now,we can get volume from central pollester. Then i think
> may be we can drop cinder volume usage.
>Cinder volume usage, do not like nova usage which is in nova
> compute. It is a cmd.  And should put it in cron. This will cause more
> work in devops. If ceilometer
> pollester can handle it. I think may be we should remove it?
>
>
>
>
>
> --
> 谦谦君子
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Duncan Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] remove gnocchi http interface?

[Jaze Lee]

2017-12-04 18:41 GMT+08:00 Julien Danjou :
> On Mon, Dec 04 2017, Jaze Lee wrote:
>
>> Right now, the dispatch will cost much time in http request. And
>> keystone auth token cost much time. Although we can configure it to
>> noauth, then anyone can update gnocchi database which is not we want.
>
>
>>  So what about to remove gnocchi http, and add a simple tcp socket, which
>>  will send to gnocchi tcp server(which will also be created.). At
>> first, we think udp,
>
> … and then "anyone can update gnocchi database which is not you want."
>
> I'm not saying it's not a good idea, but your arguments don't sound
> right. :)
>
> There's already an issue here
>   https://github.com/gnocchixyz/gnocchi/issues/496
> that you opened.
>
> At that point, you need more than "suggesting", you need to provide plan
> and code if you want that to happen "soon". :)

OK. it will be soon, wait for a little moment.. :}
>
> --
> Julien Danjou
> /* Free Software hacker
>https://julien.danjou.info */



-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] remove gnocchi http interface?

[Julien Danjou]

On Mon, Dec 04 2017, Jaze Lee wrote:

> Right now, the dispatch will cost much time in http request. And
> keystone auth token cost much time. Although we can configure it to
> noauth, then anyone can update gnocchi database which is not we want.


>  So what about to remove gnocchi http, and add a simple tcp socket, which
>  will send to gnocchi tcp server(which will also be created.). At
> first, we think udp,

… and then "anyone can update gnocchi database which is not you want."

I'm not saying it's not a good idea, but your arguments don't sound
right. :)

There's already an issue here
  https://github.com/gnocchixyz/gnocchi/issues/496
that you opened.

At that point, you need more than "suggesting", you need to provide plan
and code if you want that to happen "soon". :)

-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][cinder]about cinder volume usage

[Jaze Lee]

Hello,

Right now,we can get volume from central pollester. Then i think
may be we can drop cinder volume usage.
   Cinder volume usage, do not like nova usage which is in nova
compute. It is a cmd.  And should put it in cron. This will cause more
work in devops. If ceilometer
pollester can handle it. I think may be we should remove it?





-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] remove gnocchi http interface?

[Jaze Lee]

Hello,

Right now, the dispatch will cost much time in http request. And
keystone auth token cost much time. Although we can configure it to
noauth, then anyone can update gnocchi database which is not we want.
 So what about to remove gnocchi http, and add a simple tcp socket, which
 will send to gnocchi tcp server(which will also be created.). At
first, we think udp,
but udp can not fully utilize loadblance. So we suggest to change it to tcp.





-- 
谦谦君子

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about workload partition

[李田清]

On 2017-12-01 05:03 AM, 李田清 wrote:
>> Hello,
>>   we test workload partition, and find it's much slower than not 
>> using it.
>>   After some review, we find that, after get samples from 
>> notifications.sample
>>   ceilometer unpacks them and sends them one by one to the pipe
>>   ceilometer.pipe.*, this will make the consumer slow. Right now, 
>> the rabbit_qos_prefetch_count to 1. If we sent it to 10, the connection 
>> will be reset

> currently, i believe rabbit_qos_prefetch_count will be set to whatever 
> value you set batch_size to.
You mean in the past, it can not be set to whatever?
We test newton, and find under 1k vm, if we open workload partition,
it will be reset regularly. 

>>   regularly. Under this pos, the consumer will be very slow in 
>> workload partition. If you do not use workload partition, the messages 
>> can all be consumer. If you use it, the messages in pipe will be piled 
>> up more and more。

> what is "pos"? i'm not sure it means the same thing to both of us... or 
> well i guess it could :)
sorry, it is qos. this qos. 

>>  May be right now workload partition is not a good choice? Or any 
>> suggestion?
>> 

>i'll give a two part answer but first i'll start with a question: what 
>version of oslo.messaging do you have?

newton 5.10.2

>i see a performance drop as well but the reason for it is because of an 
>oslo.messaging bug introduced into master/pike/ocata releases. more 
>details can be found here: 
>https://bugs.launchpad.net/oslo.messaging/+bug/1734788. we're working >on 
>backporting it. we've also done some work regarding performance/memory 
>to shrink memory usage of partitioning in master[1].

>with that said, there are only two scenarios where you should have 
>partitioning enabled. if you have multiple notification agents AND:

>1. you have transformations in your pipeline
>2. you want to batch efficiently to gnocchi

>if you don't have workload partitioning on, your transform metrics will 
>probably be wrong or missing values. it also won't batch to gnocchi so 
>you'll see a lot more http requests there.

>so yes, you do have a choice to disable it, but the above is your tradeoff.

>[1] 
>https://review.openstack.org/#/q/topic:plugin+
>(status:open+OR+status:merged)+project:openstack/ceilometer

>cheers,

>-- 
>gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] about workload partition

[gordon chung]

On 2017-12-01 05:03 AM, 李田清 wrote:
> Hello,
>       we test workload partition, and find it's much slower than not 
> using it.
>       After some review, we find that, after get samples from 
> notifications.sample
>       ceilometer unpacks them and sends them one by one to the pipe
>       ceilometer.pipe.*, this will make the consumer slow. Right now, 
> the rabbit_qos_prefetch_count to 1. If we sent it to 10, the connection 
> will be reset

currently, i believe rabbit_qos_prefetch_count will be set to whatever 
value you set batch_size to.

>       regularly. Under this pos, the consumer will be very slow in 
> workload partition. If you do not use workload partition, the messages 
> can all be consumer. If you use it, the messages in pipe will be piled 
> up more and more。

what is "pos"? i'm not sure it means the same thing to both of us... or 
well i guess it could :)

>      May be right now workload partition is not a good choice? Or any 
> suggestion?
> 

i'll give a two part answer but first i'll start with a question: what 
version of oslo.messaging do you have?

i see a performance drop as well but the reason for it is because of an 
oslo.messaging bug introduced into master/pike/ocata releases. more 
details can be found here: 
https://bugs.launchpad.net/oslo.messaging/+bug/1734788. we're working on 
backporting it. we've also done some work regarding performance/memory 
to shrink memory usage of partitioning in master[1].

with that said, there are only two scenarios where you should have 
partitioning enabled. if you have multiple notification agents AND:

1. you have transformations in your pipeline
2. you want to batch efficiently to gnocchi

if you don't have workload partitioning on, your transform metrics will 
probably be wrong or missing values. it also won't batch to gnocchi so 
you'll see a lot more http requests there.

so yes, you do have a choice to disable it, but the above is your tradeoff.

[1] 
https://review.openstack.org/#/q/topic:plugin+(status:open+OR+status:merged)+project:openstack/ceilometer

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] about workload partition

[李田清]

Hello,
 we test workload partition, and find it's much slower than not using it.
 After some review, we find that, after get samples from 
notifications.sample
 ceilometer unpacks them and sends them one by one to the pipe 
 ceilometer.pipe.*, this will make the consumer slow. Right now, the 
rabbit_qos_prefetch_count to 1. If we sent it to 10, the connection will be 
reset
 regularly. Under this pos, the consumer will be very slow in workload 
partition. If you do not use workload partition, the messages can all be 
consumer. If you use it, the messages in pipe will be piled up more and more。
May be right now workload partition is not a good choice? Or any suggestion?__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Retiring ceilometerclient

[gordon chung]

On 2017-11-22 04:18 AM, Julien Danjou wrote:
> Hi,
> 
> Now that the Ceilometer API is gone, we really don't need
> ceilometerclient anymore. I've proposed a set of patches to retire it:
> 
>https://review.openstack.org/#/c/522183/
> 

just for reference, we had a super short discussion on this[1]. 
basically everything we plan on doing to interact with ceilometer (ie. 
monitoring ceilometer services) will leverage something completely new 
when it does get implemented. reusing the current client would just be 
bloat.

[1] 
http://eavesdrop.openstack.org/irclogs/%23openstack-telemetry/%23openstack-telemetry.2017-11-23.log.html#t2017-11-23T20:55:22

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] Retiring ceilometerclient

[Julien Danjou]

Hi,

Now that the Ceilometer API is gone, we really don't need
ceilometerclient anymore. I've proposed a set of patches to retire it:

  https://review.openstack.org/#/c/522183/

-- 
Julien Danjou
;; Free Software hacker
;; https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' publisher: Compression and CSV file format

[gordon chung]

On 2017-11-16 03:29 PM, Waines, Greg wrote:
> Ok ... if you don’t mind, i might create a blueprint anyways ... for 
> traceability and documentation purposes.
> 
> We/WindRiver tracks both blueprints and code submissions that we have 
> worked on, as an indication of our upstream contributions.
>
as you wish, as long as you're aware that no one reads it :). we do 
track features under bugs in launchpad[1] and just mark them as wishlist 
normally but i'm not going to stop you from creating a bp. you'll just 
need to ping someone to mark it as implemented when the time comes.

[1] https://bugs.launchpad.net/ceilometer/+bugs?orderby=-id=0

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' publisher: Compression and CSV file format

[Waines, Greg]

Ok ... if you don’t mind, i might create a blueprint anyways ... for 
traceability and documentation purposes.
We/WindRiver tracks both blueprints and code submissions that we have worked 
on, as an indication of our upstream contributions.

Greg.


From: gordon chung <g...@live.ca>
Reply-To: "openstack-dev@lists.openstack.org" 
<openstack-dev@lists.openstack.org>
Date: Thursday, November 16, 2017 at 2:59 PM
To: "openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org>
Subject: Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' 
publisher: Compression and CSV file format



On 2017-11-16 02:48 PM, Waines, Greg wrote:
Cool ... i will go ahead and submit a blueprint for these.

just an fyi, we don't use blueprints in Telemetry projects. just submit
the code :)

if there's many alternatives to the solution or if it's a something
controversial, we usually discuss it here on ML. in this case, it seems
like a pretty regular feature addition.

Wrt the compression blueprint,  locally we subclassed the base python
rotating file handler class in order to optionally compress the file as
part of the “shouldRollover” stage.

sounds good.

--
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' publisher: Compression and CSV file format

[gordon chung]

On 2017-11-16 02:48 PM, Waines, Greg wrote:
> Cool ... i will go ahead and submit a blueprint for these.

just an fyi, we don't use blueprints in Telemetry projects. just submit 
the code :)

if there's many alternatives to the solution or if it's a something 
controversial, we usually discuss it here on ML. in this case, it seems 
like a pretty regular feature addition.

> 
> Wrt the compression blueprint,  locally we subclassed the base python 
> rotating file handler class in order to optionally compress the file as 
> part of the “shouldRollover” stage.
> 

sounds good.

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' publisher: Compression and CSV file format

[Waines, Greg]

Cool ... i will go ahead and submit a blueprint for these.

Wrt the compression blueprint,  locally we subclassed the base python rotating 
file handler class in order to optionally compress the file as part of the 
“shouldRollover” stage.

Greg.


From: gordon chung <g...@live.ca>
Reply-To: "openstack-dev@lists.openstack.org" 
<openstack-dev@lists.openstack.org>
Date: Thursday, November 16, 2017 at 1:49 PM
To: "openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org>
Subject: Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' 
publisher: Compression and CSV file format



On 2017-11-16 12:44 PM, Waines, Greg wrote:
1.Compression of files (within file rotation) to minimize disk usage and
minimize bandwidth in FTP-ing these files off server
oi.e. as part of file rotation, the newly rotated file would be gzip’d
to compress the file
othis would be an optional parameter in the pipeline definition in
/etc/ceilometer/pipeline.yaml
e.g.
name: meter_file
meters:
  - "*"
publishers:
  - file:///var/test?max_bytes=1000_count=5=true
oagain,
minimizing disk usage and minimizing network bandwidth in FTP-ing these
files off server.

seems like a good idea. just curious, how would you implement this?

2.CSV (Comma Separated Values) File Format
oCSV is widely supported for many off-box Statistical Analysis-type
Tools; especially in Telecom,
oagain this would be done as an optional argument to the file publisher
§the existing format (python dictionary print) would remain as the
default format
§‘format=csv’ would be the argument added to the publisher line in order
to enable CSV format
e.g.
name: meter_file
meters:
  - "*"
publishers:
  - file:///var/test?max_bytes=1000_count=5=csv

this sounds like a good idea. i believe there was a bug previously that
suggested cleaning up the output of the file publisher/dispatcher so
it's not just dumping random blurbs of json. i think the proposed
format=csv trigger makes sense.

I think these would be valuable addons to the Ceilometer File Publisher,

agreed! thanks for suggesting them. this is me assuming you (or a
colleague) will implement this :)

--
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' publisher: Compression and CSV file format

[gordon chung]

On 2017-11-16 12:44 PM, Waines, Greg wrote:
> 
> 1.Compression of files (within file rotation) to minimize disk usage and 
> minimize bandwidth in FTP-ing these files off server
> 
> oi.e. as part of file rotation, the newly rotated file would be gzip’d 
> to compress the file
> 
> othis would be an optional parameter in the pipeline definition in 
> /etc/ceilometer/pipeline.yaml
> e.g.
> 
> name: meter_file
> 
> meters:
> 
>      - "*"
> 
> publishers:
> 
>      - file:///var/test?max_bytes=1000_count=5=true
> 
> oagain,
> minimizing disk usage and minimizing network bandwidth in FTP-ing these 
> files off server.

seems like a good idea. just curious, how would you implement this?

> 
> 2.CSV (Comma Separated Values) File Format
> 
> oCSV is widely supported for many off-box Statistical Analysis-type 
> Tools; especially in Telecom,
> 
> oagain this would be done as an optional argument to the file publisher
> 
> §the existing format (python dictionary print) would remain as the 
> default format
> 
> §‘format=csv’ would be the argument added to the publisher line in order 
> to enable CSV format
> e.g.
> 
> name: meter_file
> 
> meters:
> 
>      - "*"
> 
> publishers:
> 
>      - file:///var/test?max_bytes=1000_count=5=csv
>
this sounds like a good idea. i believe there was a bug previously that 
suggested cleaning up the output of the file publisher/dispatcher so 
it's not just dumping random blurbs of json. i think the proposed 
format=csv trigger makes sense.

> 
> I think these would be valuable addons to the Ceilometer File Publisher,
> 

agreed! thanks for suggesting them. this is me assuming you (or a 
colleague) will implement this :)

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] Blueprint proposals for 'file' publisher: Compression and CSV file format

[Waines, Greg]

Hey there,

I am looking for some feedback on 2x enhancement / blueprint proposals for the 
‘file’ publisher in ceilometer.
i.e.   
https://github.com/openstack/ceilometer/blob/master/ceilometer/publisher/file.py


Are the Ceilometer owners open to looking at blueprints for the following ?


1.   Compression of files (within file rotation) to minimize disk usage and 
minimize bandwidth in FTP-ing these files off server

oi.e. as part of file rotation, the newly rotated file would be gzip’d to 
compress the file

othis would be an optional parameter in the pipeline definition in 
/etc/ceilometer/pipeline.yaml
e.g.
name: meter_file
meters:
- "*"
publishers:
- file:///var/test?max_bytes=1000_count=5=true

oagain,
minimizing disk usage and minimizing network bandwidth in FTP-ing these files 
off server.


2.   CSV (Comma Separated Values) File Format

oCSV is widely supported for many off-box Statistical Analysis-type Tools; 
especially in Telecom,

oagain this would be done as an optional argument to the file publisher

§  the existing format (python dictionary print) would remain as the default 
format

§  ‘format=csv’ would be the argument added to the publisher line in order to 
enable CSV format
e.g.
name: meter_file
meters:
- "*"
publishers:
- file:///var/test?max_bytes=1000_count=5=csv




let me know your feedback,

I think these would be valuable addons to the Ceilometer File Publisher,

Greg.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][oslo_messaging] Random Connection reset by peer

[李田清]

Hello,
I test ceilometer agent notification workload partition. I found it is too 
fragile. 
The load is 1k cirrors vm. I make processing queue to 4 and workers to 1. 
I can sure the network is ok. But in the ceilometer agent notification log 
i see this
http://paste.openstack.org/show/624795/ 


   I am sure rabbitmq resource is enough. the status is here
http://paste.openstack.org/show/624796/


   And after some dig in i found this.
https://stackoverflow.com/questions/383738/104-connection-reset-by-peer-socket-error-or-when-does-closing-a-socket-resu


Can someone join me to test it? Or is there someone know this why and how 
to fix this?
Thanks a lot a lot.__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the workload partition willcauseconsumer disappeared

[李田清]

I  use 5.10.2 




> I test newton 5.10.2, and in ceilometer agent notification, the log shows
> 2017-10-21 03:33:19.779 225636 ERROR root [-] Unexpected exception
> occurred 60 time(s)... retrying.
> 2017-10-21 03:33:19.779 225636 ERROR root Traceback (most recent call last):
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 250, in
> wrapper
> 2017-10-21 03:33:19.779 225636 ERROR root return infunc(*args, **kwargs)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/base.py", line
> 203, in _runner
> 2017-10-21 03:33:19.779 225636 ERROR root batch_size=self.batch_size,
> batch_timeout=self.batch_timeout)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/base.py", line
> 52, in wrapper
> 2017-10-21 03:33:19.779 225636 ERROR root msg = func(in_self,
> timeout=timeout_watch.leftover(True))
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
> 286, in poll
> 2017-10-21 03:33:19.779 225636 ERROR root
> self._message_operations_handler.process()
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
> 89, in process
> 2017-10-21 03:33:19.779 225636 ERROR root task()
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py",
> line 251, in acknowledge
> 2017-10-21 03:33:19.779 225636 ERROR root self._raw_message.ack()
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/kombu/message.py", line 88, in ack
> 2017-10-21 03:33:19.779 225636 ERROR root
> self.channel.basic_ack(self.delivery_tag)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1583, in basic_ack
> 2017-10-21 03:33:19.779 225636 ERROR root self._send_method((60, 80), args)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 50, in
> _send_method
> 2017-10-21 03:33:19.779 225636 ERROR root raise
> RecoverableConnectionError('connection already closed')
> 2017-10-21 03:33:19.779 225636 ERROR root RecoverableConnectionError:
> connection already closed

what's the oslo.messaging version? i feel like this is something on 
oslo.messaging and it does sound familiar to possibly it's fixed.

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the workload partition will causeconsumer disappeared

[gordon chung]

On 25/10/17 03:25 AM, 李田清 wrote:
> I test newton 5.10.2, and in ceilometer agent notification, the log shows
> 2017-10-21 03:33:19.779 225636 ERROR root [-] Unexpected exception
> occurred 60 time(s)... retrying.
> 2017-10-21 03:33:19.779 225636 ERROR root Traceback (most recent call last):
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 250, in
> wrapper
> 2017-10-21 03:33:19.779 225636 ERROR root return infunc(*args, **kwargs)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/base.py", line
> 203, in _runner
> 2017-10-21 03:33:19.779 225636 ERROR root batch_size=self.batch_size,
> batch_timeout=self.batch_timeout)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/base.py", line
> 52, in wrapper
> 2017-10-21 03:33:19.779 225636 ERROR root msg = func(in_self,
> timeout=timeout_watch.leftover(True))
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
> 286, in poll
> 2017-10-21 03:33:19.779 225636 ERROR root
> self._message_operations_handler.process()
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
> 89, in process
> 2017-10-21 03:33:19.779 225636 ERROR root task()
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py",
> line 251, in acknowledge
> 2017-10-21 03:33:19.779 225636 ERROR root self._raw_message.ack()
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/kombu/message.py", line 88, in ack
> 2017-10-21 03:33:19.779 225636 ERROR root
> self.channel.basic_ack(self.delivery_tag)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1583, in basic_ack
> 2017-10-21 03:33:19.779 225636 ERROR root self._send_method((60, 80), args)
> 2017-10-21 03:33:19.779 225636 ERROR root File
> "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 50, in
> _send_method
> 2017-10-21 03:33:19.779 225636 ERROR root raise
> RecoverableConnectionError('connection already closed')
> 2017-10-21 03:33:19.779 225636 ERROR root RecoverableConnectionError:
> connection already closed

what's the oslo.messaging version? i feel like this is something on 
oslo.messaging and it does sound familiar to possibly it's fixed.

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the workload partition will causeconsumer disappeared

[李田清]

On 2017-10-23 09:57 PM, 李田清 wrote:
>   We test ceilometer workload partition, and find even with one 
> rabbitmq server, the ceilometer-pipe
>   will lost its consumers. Does anyone know this?
>   I configure, batch_size =1, batch_timeout =1, 
> and pipeline_processing_queues = 1.
>   If anyone know this, please point it out. Thanks
and you see no errors in notification-agent? does it start with a 
consumer or is there never a consumer?


The error is 
I test newton 5.10.2, and in ceilometer agent notification, the log shows
2017-10-21 03:33:19.779 225636 ERROR root [-] Unexpected exception occurred 60 
time(s)... retrying.
2017-10-21 03:33:19.779 225636 ERROR root Traceback (most recent call last):
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 250, in wrapper
2017-10-21 03:33:19.779 225636 ERROR root return infunc(*args, **kwargs)
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/base.py", line 203, 
in _runner
2017-10-21 03:33:19.779 225636 ERROR root batch_size=self.batch_size, 
batch_timeout=self.batch_timeout)
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/base.py", line 52, in 
wrapper
2017-10-21 03:33:19.779 225636 ERROR root msg = func(in_self, 
timeout=timeout_watch.leftover(True))
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 
286, in poll
2017-10-21 03:33:19.779 225636 ERROR root 
self._message_operations_handler.process()
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 
89, in process
2017-10-21 03:33:19.779 225636 ERROR root task()
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 
251, in acknowledge
2017-10-21 03:33:19.779 225636 ERROR root self._raw_message.ack()
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/kombu/message.py", line 88, in ack
2017-10-21 03:33:19.779 225636 ERROR root 
self.channel.basic_ack(self.delivery_tag)
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/amqp/channel.py", line 1583, in basic_ack
2017-10-21 03:33:19.779 225636 ERROR root self._send_method((60, 80), args)
2017-10-21 03:33:19.779 225636 ERROR root File 
"/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 50, in 
_send_method
2017-10-21 03:33:19.779 225636 ERROR root raise 
RecoverableConnectionError('connection already closed')
2017-10-21 03:33:19.779 225636 ERROR root RecoverableConnectionError: 
connection already closed

are you setting pipeline_processing_queues =1 as a test? because it sort 
of defeats purpose.


Yes, i just test the workload partition. If processing queues = 4, there will 
be more
partition for each pipeline 

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] the workload partition will cause consumer disappeared

[gordon chung]

On 2017-10-23 09:57 PM, 李田清 wrote:
>       We test ceilometer workload partition, and find even with one 
> rabbitmq server, the ceilometer-pipe
>       will lost its consumers. Does anyone know this?
>       I configure, batch_size =1, batch_timeout =1, 
> and pipeline_processing_queues = 1.
>       If anyone know this, please point it out. Thanks
and you see no errors in notification-agent? does it start with a 
consumer or is there never a consumer?

are you setting pipeline_processing_queues=1 as a test? because it sort 
of defeats purpose.

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] the workload partition will cause consumer disappeared

[李田清]

Hello,
 We test ceilometer workload partition, and find even with one rabbitmq 
server, the ceilometer-pipe
 will lost its consumers. Does anyone know this?
 I configure, batch_size =1, batch_timeout =1, and 
pipeline_processing_queues = 1.
 If anyone know this, please point it out. Thanks__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Time for API removal?

[gordon chung]

adding ops list

On 2017-10-13 11:01 AM, Julien Danjou wrote:
> Hey there,
> 
> We deprecated the Ceilometer API last year (Ocata) and our latest user
> survery shows than more than 50% of our users are now using Gnocchi or
> something else than the old deprecated storage methods.

thanks to all those that moved off legacy storage.

> 
> I'm starting to think it's time to stop carrying this dead code around.
> The biggest reason, other than cleaning our code base, is to stop having
> confusing options such as, e.g., "time-to-live" that make users being
> confused about where to configure the expiration of their metrics.
> 
> The question, is there any compelling reason to keep this deprecated
> stuffs for one more cycle?
> 
i'm ok with removing it. the amount of questions i've fielded related to 
Ceilometer API has dropped over the year we've officially deprecated 
it[1]. more importantly, we've had arguably no non-trivial patches to 
improve this legacy code for over 2 years so it's probably best to 
remove it so any new users don't read an old doc and attempt to install 
ceilometer storage.

i just checked docs and the install guide docs back to newton already 
tell people to use Gnocchi or something else.

tl;dr let's burn this down unless some hidden figure who's contributing 
speaks up.

[1] 
https://github.com/openstack/ceilometer/commit/6616a714009a80c7484fa2292c2331868617cb9c#diff-074c02939ac2ca794f659d87981d2220

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] Time for API removal?

[Julien Danjou]

Hey there,

We deprecated the Ceilometer API last year (Ocata) and our latest user
survery shows than more than 50% of our users are now using Gnocchi or
something else than the old deprecated storage methods.

I'm starting to think it's time to stop carrying this dead code around.
The biggest reason, other than cleaning our code base, is to stop having
confusing options such as, e.g., "time-to-live" that make users being
confused about where to configure the expiration of their metrics.

The question, is there any compelling reason to keep this deprecated
stuffs for one more cycle?

-- 
Julien Danjou
# Free Software hacker
# https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[Mohan Kumar]

Hi Rajeev,

No , We haven't started any design .

Let me know If I can help you anywhere . We can collaborate on this :)

Thanks.,
Mohankumar.N



On Tue, Aug 8, 2017 at 12:15 PM, rajeev.satyanaray...@wipro.com <
rajeev.satyanaray...@wipro.com> wrote:

> Hi MohanKumar,
>
>
>
> Thanks for the reply.
>
>
>
> Is the design part for the “OAM Operation Manager for SFC” done? What is
> the approach being used to monitor the SFC? If the design is already done,
> I would like to contribute in implementing it. Please provide your comments.
>
>
>
> Thanks and Regards,
>
> Rajeev
>
>
>
> *From:* Mohan Kumar [mailto:nmohankumar1...@gmail.com]
> *Sent:* Friday, July 28, 2017 1:32 PM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> *Cc:* Rajeev Satyanarayana (MFG & Tech) <rajeev.satyanaray...@wipro.com>
> *Subject:* ** Newsletter/Marketing email** Re: [openstack-dev]
> [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC
>
>
>
> ** This mail has been sent from an external source. Treat hyperlinks and
> attachments in this email with caution**
>
> Hi Rajeev,
>
>
>
> The Chain Monitoring is the missing piece in networking-sfc . Vikram and
> myself were discussed to introduce "SFC Manager" [1] (basically OAM tool )
> few cycle before.
>
>
>
> It'll continuously monitor an existing SFC chain,  when any SF VM goes
> down or any packet drop. It'll trigger alert and capture corresponding logs
> . But we haven't started any implementation on this. Feel free to discuss
> with community, IMO  this feature will be great addition to networking-sfc
>
>
>
> [1]  https://bugs.launchpad.net/networking-sfc/+bug/1513368
>
> <https://clicktime.symantec.com/a/1/oyvdp2ozE_Pe2sKTM51_kVGy1ODcQYS_L50v9C-IyKo=?d=OZb6_jQ3JlDcR2AISJ1S1yWSl9wA-da9R6WweBZh8-1xFSm7MxSA9a4SYoOw7U5fTTE8TgbqfudjFvUJGjDkrnVR8mWytf4M3eN4AD71w1P20EuGgVBYqX2e4XU1RLCLPuDzEem8Pvb7DaVjt_oEU15hkt_CfSTEUKh58nKrnbrfyy9rDd5XIZscEY5gl3pbQ5CoXChMOmfq68AE2WCT9a01j26KThkBjxPJbv7zay4R1vkxAxl07G4VZ26lvhz-CLOS6io8hJGjhaWkAJN_i67NzGOXXjipBTRt860ew1Njxrns15bG_IJ58TV9RRw-EMQ0HNftqhXlA28RRbuLvqfjJmivRjKHrMXjeKah9B7TvU1XTn7fAwJvIbcHA8zejHC_MUDNNpDMJQ7RfC6A0v5hl8qhiG2UCM-axWeSN5erVE652SbORAGMTScANtK7PL02O8Da9-W30pJr9e8haxOIGYVgHA%3D%3D=https%3A%2F%2Fbugs.launchpad.net%2Fnetworking-sfc%2F%2Bbug%2F1513368>
>
>
>
> IRC: #networking-sfc
>
> Weekly Meeting:  https://wiki.openstack.org/wiki/Meetings/
> ServiceFunctionChainingMeeting
> <https://clicktime.symantec.com/a/1/6dNKaHpOkefxWGyCJEXAIE-cQCSPzu0gOs-V_OdWGO4=?d=OZb6_jQ3JlDcR2AISJ1S1yWSl9wA-da9R6WweBZh8-1xFSm7MxSA9a4SYoOw7U5fTTE8TgbqfudjFvUJGjDkrnVR8mWytf4M3eN4AD71w1P20EuGgVBYqX2e4XU1RLCLPuDzEem8Pvb7DaVjt_oEU15hkt_CfSTEUKh58nKrnbrfyy9rDd5XIZscEY5gl3pbQ5CoXChMOmfq68AE2WCT9a01j26KThkBjxPJbv7zay4R1vkxAxl07G4VZ26lvhz-CLOS6io8hJGjhaWkAJN_i67NzGOXXjipBTRt860ew1Njxrns15bG_IJ58TV9RRw-EMQ0HNftqhXlA28RRbuLvqfjJmivRjKHrMXjeKah9B7TvU1XTn7fAwJvIbcHA8zejHC_MUDNNpDMJQ7RfC6A0v5hl8qhiG2UCM-axWeSN5erVE652SbORAGMTScANtK7PL02O8Da9-W30pJr9e8haxOIGYVgHA%3D%3D=https%3A%2F%2Fwiki.openstack.org%2Fwiki%2FMeetings%2FServiceFunctionChainingMeeting>
>
>
>
> Thanks.,
>
> Mohankumar.N
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Jul 27, 2017 at 11:30 AM, rajeev.satyanaray...@wipro.com <
> rajeev.satyanaray...@wipro.com> wrote:
>
> Hi Igor/Cathy/Gord,
>
>
>
> Sorry for the delay in replying.
>
>
>
> As part of monitoring the SFC, I think maintaining the details of “Number
> of flows assigned to a given SFC”, “Number of packets/bytes dropped/hit due
> to the policies at each Service Function entry/exit points or for the
> entire SFC” would be good to start with. I feel based on some of these
> details, an operator can know how much the virtual switch is loaded and
> take a call on whether to add a new SFC, or it could even help during
> debugging which service function has exactly caused the disruption to SFC.
>
> As a first step, I think it would be good to add meters to provide “number
> of packets/bytes hit due to policy at the ingress and egress of the entire
> SFC” and to realize that, we would need to add specific pollsters. We can
> use neutron_client API to fetch the ingress and egress port details and
> fetch the corresponding flows for those specific ports(also get flow_infos
> from flow_classifier and then use them to dump_flows matching those
> flow_infos) and fetch the no.of packets hit due to policy from them.
>
>
>
> I have just mentioned my idea on this. Can I please know your opinion on
> this and provide your comments?
>
>
>
> Thanks and Regards,
>
> Rajeev.
>
> The information contain

Re: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[rajeev.satyanaray...@wipro.com]

Hi MohanKumar,

Thanks for the reply.

Is the design part for the “OAM Operation Manager for SFC” done? What is the 
approach being used to monitor the SFC? If the design is already done, I would 
like to contribute in implementing it. Please provide your comments.

Thanks and Regards,
Rajeev

From: Mohan Kumar [mailto:nmohankumar1...@gmail.com]
Sent: Friday, July 28, 2017 1:32 PM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Cc: Rajeev Satyanarayana (MFG & Tech) <rajeev.satyanaray...@wipro.com>
Subject: ** Newsletter/Marketing email** Re: [openstack-dev] 
[ceilometer][networking-sfc] Meters/Statistics for Networking-SFC


** This mail has been sent from an external source. Treat hyperlinks and 
attachments in this email with caution**
Hi Rajeev,

The Chain Monitoring is the missing piece in networking-sfc . Vikram and myself 
were discussed to introduce "SFC Manager" [1] (basically OAM tool ) few cycle 
before.

It'll continuously monitor an existing SFC chain,  when any SF VM goes down or 
any packet drop. It'll trigger alert and capture corresponding logs . But we 
haven't started any implementation on this. Feel free to discuss with 
community, IMO  this feature will be great addition to networking-sfc

[1]  https://bugs.launchpad.net/networking-sfc/+bug/1513368
<https://clicktime.symantec.com/a/1/oyvdp2ozE_Pe2sKTM51_kVGy1ODcQYS_L50v9C-IyKo=?d=OZb6_jQ3JlDcR2AISJ1S1yWSl9wA-da9R6WweBZh8-1xFSm7MxSA9a4SYoOw7U5fTTE8TgbqfudjFvUJGjDkrnVR8mWytf4M3eN4AD71w1P20EuGgVBYqX2e4XU1RLCLPuDzEem8Pvb7DaVjt_oEU15hkt_CfSTEUKh58nKrnbrfyy9rDd5XIZscEY5gl3pbQ5CoXChMOmfq68AE2WCT9a01j26KThkBjxPJbv7zay4R1vkxAxl07G4VZ26lvhz-CLOS6io8hJGjhaWkAJN_i67NzGOXXjipBTRt860ew1Njxrns15bG_IJ58TV9RRw-EMQ0HNftqhXlA28RRbuLvqfjJmivRjKHrMXjeKah9B7TvU1XTn7fAwJvIbcHA8zejHC_MUDNNpDMJQ7RfC6A0v5hl8qhiG2UCM-axWeSN5erVE652SbORAGMTScANtK7PL02O8Da9-W30pJr9e8haxOIGYVgHA%3D%3D=https%3A%2F%2Fbugs.launchpad.net%2Fnetworking-sfc%2F%2Bbug%2F1513368>

IRC: #networking-sfc
Weekly Meeting:  
https://wiki.openstack.org/wiki/Meetings/ServiceFunctionChainingMeeting<https://clicktime.symantec.com/a/1/6dNKaHpOkefxWGyCJEXAIE-cQCSPzu0gOs-V_OdWGO4=?d=OZb6_jQ3JlDcR2AISJ1S1yWSl9wA-da9R6WweBZh8-1xFSm7MxSA9a4SYoOw7U5fTTE8TgbqfudjFvUJGjDkrnVR8mWytf4M3eN4AD71w1P20EuGgVBYqX2e4XU1RLCLPuDzEem8Pvb7DaVjt_oEU15hkt_CfSTEUKh58nKrnbrfyy9rDd5XIZscEY5gl3pbQ5CoXChMOmfq68AE2WCT9a01j26KThkBjxPJbv7zay4R1vkxAxl07G4VZ26lvhz-CLOS6io8hJGjhaWkAJN_i67NzGOXXjipBTRt860ew1Njxrns15bG_IJ58TV9RRw-EMQ0HNftqhXlA28RRbuLvqfjJmivRjKHrMXjeKah9B7TvU1XTn7fAwJvIbcHA8zejHC_MUDNNpDMJQ7RfC6A0v5hl8qhiG2UCM-axWeSN5erVE652SbORAGMTScANtK7PL02O8Da9-W30pJr9e8haxOIGYVgHA%3D%3D=https%3A%2F%2Fwiki.openstack.org%2Fwiki%2FMeetings%2FServiceFunctionChainingMeeting>

Thanks.,
Mohankumar.N







On Thu, Jul 27, 2017 at 11:30 AM, 
rajeev.satyanaray...@wipro.com<mailto:rajeev.satyanaray...@wipro.com> 
<rajeev.satyanaray...@wipro.com<mailto:rajeev.satyanaray...@wipro.com>> wrote:
Hi Igor/Cathy/Gord,

Sorry for the delay in replying.

As part of monitoring the SFC, I think maintaining the details of “Number of 
flows assigned to a given SFC”, “Number of packets/bytes dropped/hit due to the 
policies at each Service Function entry/exit points or for the entire SFC” 
would be good to start with. I feel based on some of these details, an operator 
can know how much the virtual switch is loaded and take a call on whether to 
add a new SFC, or it could even help during debugging which service function 
has exactly caused the disruption to SFC.
As a first step, I think it would be good to add meters to provide “number of 
packets/bytes hit due to policy at the ingress and egress of the entire SFC” 
and to realize that, we would need to add specific pollsters. We can use 
neutron_client API to fetch the ingress and egress port details and fetch the 
corresponding flows for those specific ports(also get flow_infos from 
flow_classifier and then use them to dump_flows matching those flow_infos) and 
fetch the no.of packets hit due to policy from them.

I have just mentioned my idea on this. Can I please know your opinion on this 
and provide your comments?

Thanks and Regards,
Rajeev.
The information contained in this electronic message and any attachments to 
this message are intended for the exclusive use of the addressee(s) and may 
contain proprietary, confidential or privileged information. If you are not the 
intended recipient, you should not disseminate, distribute or copy this e-mail. 
Please notify the sender immediately and destroy all copies of this message and 
any attachments. WARNING: Computer viruses can be transmitted via email. The 
recipient should check this email and any attachments for the presence of 
viruses. The company accepts no liability for any damage caused by any virus 
transmitted by this email. www.wipro.com<http://www.wipro.com>

__

Re: [openstack-dev] [ceilometer]The principle of restrict the scope of returned events

[Along Meng]

OK，thanks @gord

Look forward to other people's suggestion.

On Sat, Aug 5, 2017 at 12:12 AM, gordon chung  wrote:

> hi,
>
> i think this is best to ask this on mailing list. to be honest, i do not
> work on Panko much anymore. feel free to propose something if it doesn't
> meet your requirements.
>
> On 04/08/17 10:59 AM, Along Meng wrote:
> > HI, gord
> >
> > https://review.openstack.org/#/c/218706
> >
> > As the above patch you submitted shows that: ceilometer will restricts
> > the scope of returned events. if admin role,
> > user is allowed to query all events which have traits.project_id
> > value*that match it's own project*OR any event without any project_id
> trait.
> >
> > I think if admin role, user is allowed to query all events *no matter
> the *
> > *traits.project_id value match it's own project or not *is more property.
> >
> > So, I wonder to know why we designed the principle like your patch's
> > commit-message shows.
> > Thanks
> >
> > MengAlong
>
> --
> gord
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[Mohan Kumar]

Hi Rajeev,

The Chain Monitoring is the missing piece in networking-sfc . Vikram and
myself were discussed to introduce "SFC Manager" [1] (basically OAM tool )
few cycle before.

It'll continuously monitor an existing SFC chain,  when any SF VM goes down
or any packet drop. It'll trigger alert and capture corresponding logs .
But we haven't started any implementation on this. Feel free to discuss
with community, IMO  this feature will be great addition to networking-sfc

[1]  https://bugs.launchpad.net/networking-sfc/+bug/1513368


IRC: #networking-sfc
Weekly Meeting:
https://wiki.openstack.org/wiki/Meetings/ServiceFunctionChainingMeeting

Thanks.,
Mohankumar.N







On Thu, Jul 27, 2017 at 11:30 AM, rajeev.satyanaray...@wipro.com <
rajeev.satyanaray...@wipro.com> wrote:

> Hi Igor/Cathy/Gord,
>
>
>
> Sorry for the delay in replying.
>
>
>
> As part of monitoring the SFC, I think maintaining the details of “Number
> of flows assigned to a given SFC”, “Number of packets/bytes dropped/hit due
> to the policies at each Service Function entry/exit points or for the
> entire SFC” would be good to start with. I feel based on some of these
> details, an operator can know how much the virtual switch is loaded and
> take a call on whether to add a new SFC, or it could even help during
> debugging which service function has exactly caused the disruption to SFC.
>
> As a first step, I think it would be good to add meters to provide “number
> of packets/bytes hit due to policy at the ingress and egress of the entire
> SFC” and to realize that, we would need to add specific pollsters. We can
> use neutron_client API to fetch the ingress and egress port details and
> fetch the corresponding flows for those specific ports(also get flow_infos
> from flow_classifier and then use them to dump_flows matching those
> flow_infos) and fetch the no.of packets hit due to policy from them.
>
>
>
> I have just mentioned my idea on this. Can I please know your opinion on
> this and provide your comments?
>
>
>
> Thanks and Regards,
>
> Rajeev.
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email.
> www.wipro.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[rajeev.satyanaray...@wipro.com]

Hi Igor/Cathy/Gord,

Sorry for the delay in replying.

As part of monitoring the SFC, I think maintaining the details of "Number of 
flows assigned to a given SFC", "Number of packets/bytes dropped/hit due to the 
policies at each Service Function entry/exit points or for the entire SFC" 
would be good to start with. I feel based on some of these details, an operator 
can know how much the virtual switch is loaded and take a call on whether to 
add a new SFC, or it could even help during debugging which service function 
has exactly caused the disruption to SFC.
As a first step, I think it would be good to add meters to provide "number of 
packets/bytes hit due to policy at the ingress and egress of the entire SFC" 
and to realize that, we would need to add specific pollsters. We can use 
neutron_client API to fetch the ingress and egress port details and fetch the 
corresponding flows for those specific ports(also get flow_infos from 
flow_classifier and then use them to dump_flows matching those flow_infos) and 
fetch the no.of packets hit due to policy from them.

I have just mentioned my idea on this. Can I please know your opinion on this 
and provide your comments?

Thanks and Regards,
Rajeev.
The information contained in this electronic message and any attachments to 
this message are intended for the exclusive use of the addressee(s) and may 
contain proprietary, confidential or privileged information. If you are not the 
intended recipient, you should not disseminate, distribute or copy this e-mail. 
Please notify the sender immediately and destroy all copies of this message and 
any attachments. WARNING: Computer viruses can be transmitted via email. The 
recipient should check this email and any attachments for the presence of 
viruses. The company accepts no liability for any damage caused by any virus 
transmitted by this email. www.wipro.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer]Understandig ceilometer

[Gyorgy Szombathelyi]

Hi Gordon,

Thanks for the explanations!

Br,
György

> -Original Message-
> From: gordon chung [mailto:g...@live.ca]
> Sent: 2017 július 4, kedd 23:58
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [ceilometer]Understandig ceilometer
> 
> 
> 
> On 04/07/17 05:14 AM, Gyorgy Szombathelyi wrote:
> > - I understand that there is no collector service now. I know the old 
> > pipeline
> was: compute-agent->notification-agent->collector->storage(gnocchi).
> > Now I assumed it should be compute-agent->storage(gnocchi).
> > However I found out it is a bit different: compute-agent->notification-
> agent->storage. Is it the intended behavior or just a configuration error on
> my side?
> >
> 
> there is no storage service now. ceilometer is just collection.
> 
> compute-agent->notification-agent->gnocchi is correct. there is no way
> to publish directly from polling agents to storage currently (not without
> hacking). i'm personally not against having polling agent push to gnocchi
> directory but it's just not there currently.
> 
> > - I found no gnocchi at the publishers, but it is still there at the
> > dispatchers. How is it used as a publisher? Just with some magic
> transformation, publishers: -gnocchi:// is translated into using the direct
> publisher?
> >
> 
> we haven't ported over gnocchi to publisher. it's still a dispatcher with a
> publisher alias. there is a work item for this and it's pretty simple, we just
> didn't do it (yet). i would think it will be done in Pike since we ideally 
> want to
> remove all dispatchers in Queen.
> 
> cheers,
> 
> --
> gord
> __
> 
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer]Understandig ceilometer

[gordon chung]

On 04/07/17 05:14 AM, Gyorgy Szombathelyi wrote:
> - I understand that there is no collector service now. I know the old 
> pipeline was: compute-agent->notification-agent->collector->storage(gnocchi).
> Now I assumed it should be compute-agent->storage(gnocchi).
> However I found out it is a bit different: 
> compute-agent->notification-agent->storage. Is it the intended behavior or 
> just a configuration error on my side?
>

there is no storage service now. ceilometer is just collection.

compute-agent->notification-agent->gnocchi is correct. there is no way 
to publish directly from polling agents to storage currently (not 
without hacking). i'm personally not against having polling agent push 
to gnocchi directory but it's just not there currently.

> - I found no gnocchi at the publishers, but it is still there at the 
> dispatchers. How is it used as a publisher? Just with some magic 
> transformation, publishers: -gnocchi:// is
> translated into using the direct publisher?
>

we haven't ported over gnocchi to publisher. it's still a dispatcher 
with a publisher alias. there is a work item for this and it's pretty 
simple, we just didn't do it (yet). i would think it will be done in 
Pike since we ideally want to remove all dispatchers in Queen.

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer]Understandig ceilometer

[Gyorgy Szombathelyi]

Hi!

I try to dig into the inner workings of ceilometer, and there are things I 
couldn't figure out:

- I understand that there is no collector service now. I know the old pipeline 
was: compute-agent->notification-agent->collector->storage(gnocchi).
Now I assumed it should be compute-agent->storage(gnocchi). 
However I found out it is a bit different: 
compute-agent->notification-agent->storage. Is it the intended behavior or just 
a configuration error on my side?

- I found no gnocchi at the publishers, but it is still there at the 
dispatchers. How is it used as a publisher? Just with some magic 
transformation, publishers: -gnocchi:// is
translated into using the direct publisher?

Br,
György

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][panko] Ocata ceilometer event storage configuration

[Yaguang Tang]

 Thanks Along and Gordon , after making changes , it works.

On Fri, Jun 30, 2017 at 12:02 AM, Along Meng  wrote:

> Yes， and you need make sure ceilometer-agent-notification and panko was
> intalled in a same machine in your environment.
> Because ceilometer will load the publisher dispatcher panko from the
> module panko and use the database url[1] which configured in panko.conf to
> init the database connection.
>
> [1]
> https://github.com/openstack/panko/blob/stable/ocata/panko/
> dispatcher/database.py#L43
>
> MengAlong
>
>
> On Thu, Jun 29, 2017 at 8:56 PM, gordon chung  wrote:
>
>>
>>
>> On 29/06/17 07:24 AM, Yaguang Tang wrote:
>> > sinks:
>> > - name: event_sink
>> >   transformers:
>> >   triggers:
>> >   publishers:
>> >   - direct://
>> >   - panko://
>>
>> the publisher path is only available if you have Pike Panko. you need to
>> either backport[1] or configure your publisher as:
>>
>> direct://?dispatcher=panko
>>
>>
>> [1]
>> https://github.com/openstack/panko/commit/d785015552937455d7
>> 6f083d313a73a7c0c076b3
>>
>> cheers,
>> --
>> gord
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Tang Yaguang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][panko] Ocata ceilometer event storage configuration

[Along Meng]

Yes， and you need make sure ceilometer-agent-notification and panko was
intalled in a same machine in your environment.
Because ceilometer will load the publisher dispatcher panko from the module
panko and use the database url[1] which configured in panko.conf to init
the database connection.

[1]
https://github.com/openstack/panko/blob/stable/ocata/panko/dispatcher/database.py#L43

MengAlong


On Thu, Jun 29, 2017 at 8:56 PM, gordon chung  wrote:

>
>
> On 29/06/17 07:24 AM, Yaguang Tang wrote:
> > sinks:
> > - name: event_sink
> >   transformers:
> >   triggers:
> >   publishers:
> >   - direct://
> >   - panko://
>
> the publisher path is only available if you have Pike Panko. you need to
> either backport[1] or configure your publisher as:
>
> direct://?dispatcher=panko
>
>
> [1]
> https://github.com/openstack/panko/commit/d785015552937455d76f083d313a73
> a7c0c076b3
>
> cheers,
> --
> gord
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][panko] Ocata ceilometer event storage configuration

[gordon chung]

On 29/06/17 07:24 AM, Yaguang Tang wrote:
> sinks:
> - name: event_sink
>   transformers:
>   triggers:
>   publishers:
>   - direct://
>   - panko://

the publisher path is only available if you have Pike Panko. you need to 
either backport[1] or configure your publisher as:

direct://?dispatcher=panko


[1] 
https://github.com/openstack/panko/commit/d785015552937455d76f083d313a73a7c0c076b3

cheers,
-- 
gord

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][panko] Ocata ceilometer event storage configuration

[Yaguang Tang]

Hi all,

 I am using Ocata Ceilometer  Gnocchi with Panko, Panko is configured to
use MySQL as backend to store events, I configured Ceilometer
event_pipeline.yaml as follow:


sinks:
- name: event_sink
  transformers:
  triggers:
  publishers:
  - direct://
  - panko://

but it seems no events stored  actually, looking at the code, Ceilometer
have no event storage backend already, so how to config Ceilometer to
pushlish/dispatch events to store through Panko database ?

-- 
Tang Yaguang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] deprecating pollster-list option

[gordon chung]

hi,

i'm proposing to deprecate/remove pollster-list config option. if you're 
like me and wondering what this is, we apparently support ability to 
filter pollsters loaded by polling agent[1].

the reason i'd like to deprecate is because we already have this 
functionality in the polling|pipeline.yaml where you define what meters 
you want to enable. having the ability to do it in two different places 
causes conflict/confusion and i think the yaml path is probably the one 
people know about and is the more logical path.

disclaimer: i also want to drop the pollster-list option because there's 
a bug[2] related to it and i personally don't think it's worth fixing it.

[1] 
https://github.com/openstack/ceilometer/blob/7eb37ace7012a203ed83bd27b57f3db6f59f4547/ceilometer/cmd/polling.py#L69-L73
[2] https://bugs.launchpad.net/ceilometer/+bug/1603320

cheers,
-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[gordon chung]

On 25/06/17 08:10 AM, rajeev.satyanaray...@wipro.com wrote:
> I am interested to know if there are any meters available for monitoring
> SFC through ceilometer, like no.of flows associated with an SFC or
> packets in/out for an SFC etc?
>
> If they are available, please let me know how to configure and use them.
> If not, are there any plans of providing support to them in coming releases?
>

i'm unaware of a meter in ceilometer that does this but i'd welcome it 
if contributed. please don't hesitate to reach out if you have questions 
on adding new meters to ceilometer.

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[Cathy Zhang]

Hi Rajeev,

There is no meter yet. I think it is a good idea to add it.
Would you like to share your thought in more detail?

Thanks,
Cathy

From: Duarte Cardoso, Igor [mailto:igor.duarte.card...@intel.com]
Sent: Monday, June 26, 2017 2:37 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for 
Networking-SFC

Hi Rajeev, there are no meters as far as I know and I'm not aware of any plans 
at the moment.
What else do you have in mind in terms of monitoring?

Best regards,
Igor.

From: rajeev.satyanaray...@wipro.com<mailto:rajeev.satyanaray...@wipro.com> 
[mailto:rajeev.satyanaray...@wipro.com]
Sent: Sunday, June 25, 2017 1:10 PM
To: openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>
Subject: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for 
Networking-SFC


Hi All,



I am interested to know if there are any meters available for monitoring SFC 
through ceilometer, like no.of flows associated with an SFC or packets in/out 
for an SFC etc?

If they are available, please let me know how to configure and use them. If 
not, are there any plans of providing support to them in coming releases?



Thanking you,



Regards,

Rajeev.
The information contained in this electronic message and any attachments to 
this message are intended for the exclusive use of the addressee(s) and may 
contain proprietary, confidential or privileged information. If you are not the 
intended recipient, you should not disseminate, distribute or copy this e-mail. 
Please notify the sender immediately and destroy all copies of this message and 
any attachments. WARNING: Computer viruses can be transmitted via email. The 
recipient should check this email and any attachments for the presence of 
viruses. The company accepts no liability for any damage caused by any virus 
transmitted by this email. www.wipro.com<http://www.wipro.com>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[Duarte Cardoso, Igor]

Hi Rajeev, there are no meters as far as I know and I'm not aware of any plans 
at the moment.
What else do you have in mind in terms of monitoring?

Best regards,
Igor.

From: rajeev.satyanaray...@wipro.com [mailto:rajeev.satyanaray...@wipro.com]
Sent: Sunday, June 25, 2017 1:10 PM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for 
Networking-SFC


Hi All,



I am interested to know if there are any meters available for monitoring SFC 
through ceilometer, like no.of flows associated with an SFC or packets in/out 
for an SFC etc?

If they are available, please let me know how to configure and use them. If 
not, are there any plans of providing support to them in coming releases?



Thanking you,



Regards,

Rajeev.
The information contained in this electronic message and any attachments to 
this message are intended for the exclusive use of the addressee(s) and may 
contain proprietary, confidential or privileged information. If you are not the 
intended recipient, you should not disseminate, distribute or copy this e-mail. 
Please notify the sender immediately and destroy all copies of this message and 
any attachments. WARNING: Computer viruses can be transmitted via email. The 
recipient should check this email and any attachments for the presence of 
viruses. The company accepts no liability for any damage caused by any virus 
transmitted by this email. www.wipro.com<http://www.wipro.com>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer][networking-sfc] Meters/Statistics for Networking-SFC

[rajeev.satyanaray...@wipro.com]

Hi All,


I am interested to know if there are any meters available for monitoring SFC 
through ceilometer, like no.of flows associated with an SFC or packets in/out 
for an SFC etc?

If they are available, please let me know how to configure and use them. If 
not, are there any plans of providing support to them in coming releases?


Thanking you,


Regards,

Rajeev.

The information contained in this electronic message and any attachments to 
this message are intended for the exclusive use of the addressee(s) and may 
contain proprietary, confidential or privileged information. If you are not the 
intended recipient, you should not disseminate, distribute or copy this e-mail. 
Please notify the sender immediately and destroy all copies of this message and 
any attachments. WARNING: Computer viruses can be transmitted via email. The 
recipient should check this email and any attachments for the presence of 
viruses. The company accepts no liability for any damage caused by any virus 
transmitted by this email. www.wipro.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer]

[gordon chung]

On 09/05/17 10:08 AM, simona marinova wrote:

>  The Alarming service doesn't work at this point. For example the
> command "ceilometer alarm-list" gives the error:
>
>

you should be using aodhclient if you have aodh.

>
> Now my biggest concern is that the Alarming service database
> (MySQL-based) and the Data collection service database (MongoDB) are not
> communicating properly. Is it possible for the Aodh to access the data
> from MongoDB?

the alarming database and the deprecated ceilometer db are not suppose 
to be communicating with each other so i'm not entirely sure what you 
are expecting. ceilometer meter-list grabs data from legacy ceilometer 
db, aodh alarm list grabs data from aodh.


-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer]

[simona marinova]

Hello,


I am working on an Openstack Newton platform and I am currently trying to 
implement the Telemetry services, Ceilometer and Aodh.

I installed and configured Aodh with MySQL first and it worked. Afterwards, I 
installed Ceilometer following the latest update which includes MongoDB, unlike 
the previous version with Gnocchi.


Ceilometer now works, but only the commands which do not include Aodh give the 
correct output, for example "ceilometer meter-list", "ceilometer resource-list" 
etc.


 The Alarming service doesn't work at this point. For example the command 
"ceilometer alarm-list" gives the error:


HTTPConnectionPool(host='controller', port=8042): Max retries exceeded with 
url: /v2/alarms (Caused by 
NewConnectionError(': Failed to establish a new connection: [Errno 111] 
Connection refused',))

Now my biggest concern is that the Alarming service database (MySQL-based) and 
the Data collection service database (MongoDB) are not communicating properly. 
Is it possible for the Aodh to access the data from MongoDB?

Additionally aodh-dbsync gives error, because it cannot detect the module 
gnocchiclient. There aren't gnocchi modules involved in this version.

What kind of configuration needs to be done in order for Telemetry and Alarming 
to work properly?

Best regards,
Simona



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev