Re: [openstack-dev] introducing cyclops
On Tue, Aug 12, 2014 at 05:47:49PM +1200, Fei Long Wang wrote: Our suggestion for the first IRC meeting is 25/August 8PM-10PM UTC time on Freenodes's #openstack-rating channel. Thoughts? Please reply with the best date/time for you so we can figure out a time to start. I'll like to participate to this meeting, but one of my colleagues will not be available on the 25th. Maybe we can shift the date to the 26th? Thanks ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] introducing cyclops
I am also highly interested. A very large adoption inhibitor has been the ability to control cloud consumption with charge-back and/or cost center billing support. Would love to talk about this. *Adam Lawson* AQORN, Inc. 427 North Tatnall Street Ste. 58461 Wilmington, Delaware 19801-2230 Toll-free: (844) 4-AQORN-NOW ext. 101 International: +1 302-387-4660 Direct: +1 916-246-2072 On Tue, Aug 12, 2014 at 8:48 AM, Stephane Albert sheepr...@nullplace.com wrote: On Tue, Aug 12, 2014 at 05:47:49PM +1200, Fei Long Wang wrote: Our suggestion for the first IRC meeting is 25/August 8PM-10PM UTC time on Freenodes's #openstack-rating channel. Thoughts? Please reply with the best date/time for you so we can figure out a time to start. I'll like to participate to this meeting, but one of my colleagues will not be available on the 25th. Maybe we can shift the date to the 26th? Thanks ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] introducing cyclops
I think the situation described by Doug is what most OpenStack newcomers are facing. TBH, the company I work for(Catalyst IT Ltd, a NZ based open source company) run into the same challenge, so we had to cook something in our kitchen and are currently running it for our cloud service. The difference is we created it as a 'Rating' instead of 'Billing' system, since we understand that most companies already have something in place as the ^real^ billing system (in our case OpenERP). We see this rating project as bridge between Ceilometer and whatever systems people are using to do the billing/invoicing. All the commercial sensitive information and business rules are kept in the billing system (for example: OpenERP). The rating project only raises the 'sales orders / bills / invoices' based on the usage reported by Ceilometer on a given period. In our case this back-end is pluggable, allowing organizations to integrate with their existing ERP systems or even generate invoices in the form spreadsheets if they don't have one. Anyway, I believe there is a real requirement for a rating system in OpenStack, no? Consider these projects [1][2][3][4] and cyclops/cloudkitty mentioned in this loop. And I would like to refer the charts presented by Nicolas Barcet about Ceilometer [5], which defined something like below. Obviously, there is still a gap for rating at least(after almost 2 years), which can be an independent service or an advanced plugin/service of Ceilometer (like LBaaS for Neutron, depending on the scope). *Metering* -- Collect usage data *Rating* -- Transform usage data into billable items and calculate costs *Billing* -- Create invoice, collect payment So it seems the question is changing from '/do users need a rating system?/' to '/Would OpenStack benefit from having an out-of-box rating project?/'. I would say yes, why not. More and more companies are trying to leverage OpenStack as a public/private cloud solution and charge their customers, so it would be great if there is an out-of-box rating solution from OpenStack community. Besides, like Eoghan suggested, it would be great to have a rating/billing session in Paris . Before that, I would like to suggest some regular IRC meetings to discuss this topic to work out the goal, scope, key use cases and roadmap. Our suggestion for the first IRC meeting is 25/August 8PM-10PM UTC time on Freenodes's #openstack-rating channel. Thoughts? Please reply with the best date/time for you so we can figure out a time to start. Cheers. [1] Dough https://github.com/lzyeval/dough [2] trystack.org billing https://github.com/trystack/dash_billing [3] nova-billing https://github.com/griddynamics/nova-billing [4] billingstack https://github.com/billingstack [5] https://docs.google.com/presentation/d/1ytYhQGR5SxoccZ-wuza2n0H2mS7HniP9HoFDUpuGDiM/edit#slide=id.g1f73edb4_0_2 On 09/08/14 06:13, Doug Hellmann wrote: On Aug 8, 2014, at 3:34 AM, Piyush Harsh h...@zhaw.ch mailto:h...@zhaw.ch wrote: Dear Eoghan, Thanks for your comments. Although you are correct that rating, charging, and billing policies are commercially sensitive to the operators, still if an operator has an openstack installation, I do not see why the stack could not offer a service that supports ways for the operator to input desired policies, rules, etc to do charging and billing out of the box. These policies could still only be accessible to the operator. I think the point was more that most deployers we talked to at the beginning of the project already had tools that managed the rates and charging, but needed the usage data to feed into those tools. That was a while back, though and, as you say, it’s quite possible we have new users without similar tools in place, so I’m glad to see a couple of groups working on taking billing integration one step further. At the very least working with teams building open source consumers of ceilometer’s API will help us understand if there are any ways to make it easier to use. Doug Furthermore, one could envision that using heat together with some django magic, this could even be offered as a service for tenants of the operators who could be distributors or resellers in his client ecosystem, allowing them to set their own custom policies. I believe such stack based solution would be very much welcome by SMEs, new entrants, etc. I am planning to attend the Kilo summit in Paris, and I would be very glad to talk with you and others on this idea and on Cyclops :) Forking the codebase to stackforge is something which is definitely possible and thanks a lot for suggesting it. Looking forward to more constructive discussions on this with you and others. Kind regards, Piyush. ___ Dr. Piyush Harsh, Ph.D. Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) [Site] http://piyush-harsh.info http://piyush-harsh.info/ [Research Lab]
Re: [openstack-dev] introducing cyclops
Dear Eoghan, Thanks for your comments. Although you are correct that rating, charging, and billing policies are commercially sensitive to the operators, still if an operator has an openstack installation, I do not see why the stack could not offer a service that supports ways for the operator to input desired policies, rules, etc to do charging and billing out of the box. These policies could still only be accessible to the operator. Furthermore, one could envision that using heat together with some django magic, this could even be offered as a service for tenants of the operators who could be distributors or resellers in his client ecosystem, allowing them to set their own custom policies. I believe such stack based solution would be very much welcome by SMEs, new entrants, etc. I am planning to attend the Kilo summit in Paris, and I would be very glad to talk with you and others on this idea and on Cyclops :) Forking the codebase to stackforge is something which is definitely possible and thanks a lot for suggesting it. Looking forward to more constructive discussions on this with you and others. Kind regards, Piyush. ___ Dr. Piyush Harsh, Ph.D. Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) [Site] http://piyush-harsh.info [Research Lab] http://www.cloudcomp.ch/ Fax: +41(0)58.935.7403 GPG Keyid: 9C5A8838 On Fri, Aug 8, 2014 at 12:01 AM, Eoghan Glynn egl...@redhat.com wrote: Dear All, Let me use my first post to this list to introduce Cyclops and initiate a discussion towards possibility of this platform as a future incubated project in OpenStack. We at Zurich university of Applied Sciences have a python project in open source (Apache 2 Licensing) that aims to provide a platform to do rating-charging-billing over ceilometer. We call is Cyclops (A Charging platform for OPenStack CLouds). The initial proof of concept code can be accessed here: https://github.com/icclab/cyclops-web https://github.com/icclab/cyclops-tmanager Disclaimer: This is not the best code out there, but will be refined and documented properly very soon! A demo video from really early days of the project is here: https://www.youtube.com/watch?v=ZIwwVxqCio0 and since this video was made, several bug fixes and features were added. The idea presentation was done at Swiss Open Cloud Day at Bern and the talk slides can be accessed here: http://piyush-harsh.info/content/ocd-bern2014.pdf , and more recently the research paper on the idea was published in 2014 World Congress in Computer Science (Las Vegas), which can be accessed here: http://piyush-harsh.info/content/GCA2014-rcb.pdf I was wondering, if our effort is something that OpenStack Ceilometer/Telemetry release team would be interested in? I do understand that initially rating-charging-billing service may have been left out by choice as they would need to be tightly coupled with existing CRM/Billing systems, but Cyclops design (intended) is distributed, service oriented architecture with each component allowing for possible integration with external software via REST APIs. And therefore Cyclops by design is CRM/Billing platform agnostic. Although Cyclops PoC implementation does include a basic bill generation module. We in our team are committed to this development effort and we will have resources (interns, students, researchers) work on features and improve the code-base for a foreseeable number of years to come. Do you see a chance if our efforts could make in as an incubated project in OpenStack within Ceilometer? Hi Piyush, Thanks for bringing this up! I should preface my remarks by setting out a little OpenStack history, in terms of the original decision not to include the rating and billing stages of the pipeline under the ambit of the ceilometer project. IIUC, the logic was that such rating/billing policies were very likely to be: (a) commercially sensitive for competing cloud operators and: (b) already built-out via existing custom/proprietary systems The folks who were directly involved at the outset of ceilometer can correct me if I've misrepresented the thinking that pertained at the time. While that logic seems to still apply, I would be happy to learn more about the work you've done already on this, and would be open to hearing arguments for and against. Are you planning to attend the Kilo summit in Paris (Nov 3-7)? If so, it would be a good opportunity to discuss further in person. In the meantime, stackforge provides a low-bar-to-entry for projects in the OpenStack ecosystem that may, or may not, end up as incubated projects or as dependencies taken by graduated projects. So you might consider moving your code there? Cheers, Eoghan I really would like to hear back from you, comments, suggestions, etc. Kind regards, Piyush.
Re: [openstack-dev] introducing cyclops
On Thu, Aug 07, 2014 at 12:01:04PM +0200, Piyush Harsh wrote: Dear All, Let me use my first post to this list to introduce Cyclops and initiate a discussion towards possibility of this platform as a future incubated project in OpenStack. We at Zurich university of Applied Sciences have a python project in open source (Apache 2 Licensing) that aims to provide a platform to do rating-charging-billing over ceilometer. We call is Cyclops (A Charging platform for OPenStack CLouds). The initial proof of concept code can be accessed here: https://github.com/ icclab/cyclops-web https://github.com/icclab/cyclops-tmanager Disclaimer: This is not the best code out there, but will be refined and documented properly very soon! A demo video from really early days of the project is here: https:// www.youtube.com/watch?v=ZIwwVxqCio0 and since this video was made, several bug fixes and features were added. The idea presentation was done at Swiss Open Cloud Day at Bern and the talk slides can be accessed here: http://piyush-harsh.info/content/ocd-bern2014.pdf, and more recently the research paper on the idea was published in 2014 World Congress in Computer Science (Las Vegas), which can be accessed here: http:// piyush-harsh.info/content/GCA2014-rcb.pdf I was wondering, if our effort is something that OpenStack Ceilometer/Telemetry release team would be interested in? I do understand that initially rating-charging-billing service may have been left out by choice as they would need to be tightly coupled with existing CRM/ Billing systems, but Cyclops design (intended) is distributed, service oriented architecture with each component allowing for possible integration with external software via REST APIs. And therefore Cyclops by design is CRM/Billing platform agnostic. Although Cyclops PoC implementation does include a basic bill generation module. We in our team are committed to this development effort and we will have resources (interns, students, researchers) work on features and improve the code-base for a foreseeable number of years to come. Do you see a chance if our efforts could make in as an incubated project in OpenStack within Ceilometer? I really would like to hear back from you, comments, suggestions, etc. Kind regards, Piyush. ___ Dr. Piyush Harsh, Ph.D. Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) [Site] http://piyush-harsh.info [Research Lab] http://www.cloudcomp.ch/ Fax: +41(0)58.935.7403 GPG Keyid: 9C5A8838 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Hi, It's billing day on openstack-dev list as I can see ;) Glad to see there is other people wanting to implement Open Source billing. We've started some work around pricing and billing integration inside OpenStack, and already showed our first POC at the last summit in Atlanta to some Ceilometer devs. We are planning on releasing some major improvements today, the changes are massive so the code has been split in multiple patch/reviews and we started sending it to the gate yesterday. We've got some review in openstack-infra waiting for horizon and python client repositories. As soon as the reviews are validated we will push the code to customize Horizon dashboard to integrate with the billing modules and real time quoting on instance creation. The project source code is available at: http://github.com/stackforge/cloudkitty If you want more informations you can find a wiki page at: https://wiki.openstack.org/wiki/CloudKitty or reach us via irc on freenode #cloudkitty. Maybe we can't talk together about the future of our projects when we've done pushing changes on stackforge. Cheers, Stéphane ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] introducing cyclops
On Aug 8, 2014, at 3:34 AM, Piyush Harsh h...@zhaw.ch wrote: Dear Eoghan, Thanks for your comments. Although you are correct that rating, charging, and billing policies are commercially sensitive to the operators, still if an operator has an openstack installation, I do not see why the stack could not offer a service that supports ways for the operator to input desired policies, rules, etc to do charging and billing out of the box. These policies could still only be accessible to the operator. I think the point was more that most deployers we talked to at the beginning of the project already had tools that managed the rates and charging, but needed the usage data to feed into those tools. That was a while back, though and, as you say, it’s quite possible we have new users without similar tools in place, so I’m glad to see a couple of groups working on taking billing integration one step further. At the very least working with teams building open source consumers of ceilometer’s API will help us understand if there are any ways to make it easier to use. Doug Furthermore, one could envision that using heat together with some django magic, this could even be offered as a service for tenants of the operators who could be distributors or resellers in his client ecosystem, allowing them to set their own custom policies. I believe such stack based solution would be very much welcome by SMEs, new entrants, etc. I am planning to attend the Kilo summit in Paris, and I would be very glad to talk with you and others on this idea and on Cyclops :) Forking the codebase to stackforge is something which is definitely possible and thanks a lot for suggesting it. Looking forward to more constructive discussions on this with you and others. Kind regards, Piyush. ___ Dr. Piyush Harsh, Ph.D. Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) [Site] http://piyush-harsh.info [Research Lab] http://www.cloudcomp.ch/ Fax: +41(0)58.935.7403 GPG Keyid: 9C5A8838 On Fri, Aug 8, 2014 at 12:01 AM, Eoghan Glynn egl...@redhat.com wrote: Dear All, Let me use my first post to this list to introduce Cyclops and initiate a discussion towards possibility of this platform as a future incubated project in OpenStack. We at Zurich university of Applied Sciences have a python project in open source (Apache 2 Licensing) that aims to provide a platform to do rating-charging-billing over ceilometer. We call is Cyclops (A Charging platform for OPenStack CLouds). The initial proof of concept code can be accessed here: https://github.com/icclab/cyclops-web https://github.com/icclab/cyclops-tmanager Disclaimer: This is not the best code out there, but will be refined and documented properly very soon! A demo video from really early days of the project is here: https://www.youtube.com/watch?v=ZIwwVxqCio0 and since this video was made, several bug fixes and features were added. The idea presentation was done at Swiss Open Cloud Day at Bern and the talk slides can be accessed here: http://piyush-harsh.info/content/ocd-bern2014.pdf , and more recently the research paper on the idea was published in 2014 World Congress in Computer Science (Las Vegas), which can be accessed here: http://piyush-harsh.info/content/GCA2014-rcb.pdf I was wondering, if our effort is something that OpenStack Ceilometer/Telemetry release team would be interested in? I do understand that initially rating-charging-billing service may have been left out by choice as they would need to be tightly coupled with existing CRM/Billing systems, but Cyclops design (intended) is distributed, service oriented architecture with each component allowing for possible integration with external software via REST APIs. And therefore Cyclops by design is CRM/Billing platform agnostic. Although Cyclops PoC implementation does include a basic bill generation module. We in our team are committed to this development effort and we will have resources (interns, students, researchers) work on features and improve the code-base for a foreseeable number of years to come. Do you see a chance if our efforts could make in as an incubated project in OpenStack within Ceilometer? Hi Piyush, Thanks for bringing this up! I should preface my remarks by setting out a little OpenStack history, in terms of the original decision not to include the rating and billing stages of the pipeline under the ambit of the ceilometer project. IIUC, the logic was that such rating/billing policies were very likely to be: (a) commercially sensitive for competing cloud operators and: (b) already built-out via existing custom/proprietary systems The folks who were directly involved at the outset of ceilometer can correct me if I've misrepresented the thinking that pertained
Re: [openstack-dev] introducing cyclops
Dear All, Let me use my first post to this list to introduce Cyclops and initiate a discussion towards possibility of this platform as a future incubated project in OpenStack. We at Zurich university of Applied Sciences have a python project in open source (Apache 2 Licensing) that aims to provide a platform to do rating-charging-billing over ceilometer. We call is Cyclops (A Charging platform for OPenStack CLouds). The initial proof of concept code can be accessed here: https://github.com/icclab/cyclops-web https://github.com/icclab/cyclops-tmanager Disclaimer: This is not the best code out there, but will be refined and documented properly very soon! A demo video from really early days of the project is here: https://www.youtube.com/watch?v=ZIwwVxqCio0 and since this video was made, several bug fixes and features were added. The idea presentation was done at Swiss Open Cloud Day at Bern and the talk slides can be accessed here: http://piyush-harsh.info/content/ocd-bern2014.pdf , and more recently the research paper on the idea was published in 2014 World Congress in Computer Science (Las Vegas), which can be accessed here: http://piyush-harsh.info/content/GCA2014-rcb.pdf I was wondering, if our effort is something that OpenStack Ceilometer/Telemetry release team would be interested in? I do understand that initially rating-charging-billing service may have been left out by choice as they would need to be tightly coupled with existing CRM/Billing systems, but Cyclops design (intended) is distributed, service oriented architecture with each component allowing for possible integration with external software via REST APIs. And therefore Cyclops by design is CRM/Billing platform agnostic. Although Cyclops PoC implementation does include a basic bill generation module. We in our team are committed to this development effort and we will have resources (interns, students, researchers) work on features and improve the code-base for a foreseeable number of years to come. Do you see a chance if our efforts could make in as an incubated project in OpenStack within Ceilometer? Hi Piyush, Thanks for bringing this up! I should preface my remarks by setting out a little OpenStack history, in terms of the original decision not to include the rating and billing stages of the pipeline under the ambit of the ceilometer project. IIUC, the logic was that such rating/billing policies were very likely to be: (a) commercially sensitive for competing cloud operators and: (b) already built-out via existing custom/proprietary systems The folks who were directly involved at the outset of ceilometer can correct me if I've misrepresented the thinking that pertained at the time. While that logic seems to still apply, I would be happy to learn more about the work you've done already on this, and would be open to hearing arguments for and against. Are you planning to attend the Kilo summit in Paris (Nov 3-7)? If so, it would be a good opportunity to discuss further in person. In the meantime, stackforge provides a low-bar-to-entry for projects in the OpenStack ecosystem that may, or may not, end up as incubated projects or as dependencies taken by graduated projects. So you might consider moving your code there? Cheers, Eoghan I really would like to hear back from you, comments, suggestions, etc. Kind regards, Piyush. ___ Dr. Piyush Harsh, Ph.D. Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) [Site] http://piyush-harsh.info [Research Lab] http://www.cloudcomp.ch/ Fax: +41(0)58.935.7403 GPG Keyid: 9C5A8838 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] introducing cyclops
Hi, are you on IRC? :) Endre 2014-08-07 12:01 GMT+02:00 Piyush Harsh h...@zhaw.ch: Dear All, Let me use my first post to this list to introduce Cyclops and initiate a discussion towards possibility of this platform as a future incubated project in OpenStack. We at Zurich university of Applied Sciences have a python project in open source (Apache 2 Licensing) that aims to provide a platform to do rating-charging-billing over ceilometer. We call is Cyclops (A Charging platform for OPenStack CLouds). The initial proof of concept code can be accessed here: https://github.com/icclab/cyclops-web https://github.com/icclab/cyclops-tmanager *Disclaimer: This is not the best code out there, but will be refined and documented properly very soon!* A demo video from really early days of the project is here: https://www.youtube.com/watch?v=ZIwwVxqCio0 and since this video was made, several bug fixes and features were added. The idea presentation was done at Swiss Open Cloud Day at Bern and the talk slides can be accessed here: http://piyush-harsh.info/content/ocd-bern2014.pdf, and more recently the research paper on the idea was published in 2014 World Congress in Computer Science (Las Vegas), which can be accessed here: http://piyush-harsh.info/content/GCA2014-rcb.pdf I was wondering, if our effort is something that OpenStack Ceilometer/Telemetry release team would be interested in? I do understand that initially rating-charging-billing service may have been left out by choice as they would need to be tightly coupled with existing CRM/Billing systems, but Cyclops design (intended) is distributed, service oriented architecture with each component allowing for possible integration with external software via REST APIs. And therefore Cyclops by design is CRM/Billing platform agnostic. Although Cyclops PoC implementation does include a basic bill generation module. We in our team are committed to this development effort and we will have resources (interns, students, researchers) work on features and improve the code-base for a foreseeable number of years to come. Do you see a chance if our efforts could make in as an incubated project in OpenStack within Ceilometer? I really would like to hear back from you, comments, suggestions, etc. Kind regards, Piyush. ___ Dr. Piyush Harsh, Ph.D. Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) [Site] http://piyush-harsh.info [Research Lab] http://www.cloudcomp.ch/ Fax: +41(0)58.935.7403 GPG Keyid: 9C5A8838 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
