RE: [VOTE] Accept Lens into the Apache Incubator (earlier called Grill)
+1 (non-binding) Srikanth Sundarrajan Date: Mon, 6 Oct 2014 17:21:41 +0530 Subject: [VOTE] Accept Lens into the Apache Incubator (earlier called Grill) From: sha...@apache.org To: general@incubator.apache.org Following the discussion earlier in the thread https://www.mail-archive.com/general@incubator.apache.org/msg45208.html I would like to call a Vote for accepting Lens as a new incubator project. The proposal is available at: https://wiki.apache.org/incubator/LensProposal Vote is open till Oct 09, 2014 4 PM PST. [ ] +1 accept Lens in the Incubator [ ] +/-0 [ ] -1 because... Only Votes from Incubator PMC members are binding, but all are welcome to express their thoughts. I am +1 (non-binding). Thanks Sharad
Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
Hi, we have been allowed to open-source one of our company internal projects - currently called Bifroest. Bifroest is a storage backend for graphite-web, based on Apache Cassandra. I'm quite happy about this, and now I'm in the process of finding the best options and means to do so. This mail isn't an entire proposal yet, but I will try to stick at least to the major points in a proposal. What does Bifroest do, and where does it come from. At GoodgameStudios, we used Munin for most of our monitoring, using a lot of custom plugins for our servers and pushing 500 - 700 hosts around. That's ambitious with munin and by now, the munin-master is not able to take the stress anymore. As such, we started to evaluate graphite, since graphite is the state of the art larger scale monitoring solution. To start evaluating graphite, we deployed graphite with a carbon backend on a virtual machine. Our senior monitoring admin (which we didn't have back then) probably just had to giggle a bit and doesn't know why - things didn't perform that well on a virtual machine. It could handle the important data, but the system didn't seem to scale that well. An admin would have tossed hardware at this, SSD-Raids and all that, naturally. But we are software engineers, not admins, thus we tossed software at it (until we required hardware) :) Our intention was to have a graphite with data stored in a distributed database. A distributed database would scale both in storage space and in load the system can deal with. And it's all behind a well-defined interface. That seemed like a nifty feature for a scalable monitoring system. Hence, we tried Cyanide, since Cyanide was just that. Tossed a lot of data into Apache Cassandra, click on the metric tree and... well. Nothing happened, since Cyanide figured that a select * across several 100k rows is a grand idea. After that, we looked at InfluxDB, but at the time we started developing this, InfluxDB didn't support data aggregation and seemed to be in a very, very early stage of development. Thus, the first thought of bifroest was born: Why don't we take the good parts of Cyanide, a solid distributed database, such as Apache Cassandra, and the good parts of carbon and toss them in a big stew? That's what we did, and that's what we are currently deploying as our productive monitoring system, graphite on bifroest as a frontend for apache cassandra. Fun features of this system include: - Existing graphite and most carbon apis: -- Full support of the graphite rest API, since we are just a backend. -- Support for the Plaintext Protocol of Carbon -- Planned: An AMQP interface to handle globally distributed networks - Neat things, which graphite could do as well: -- A fast key cache -- A fast value-cache, which is fed by the data collection to hit the database as little as possible - New things, Graphite+carbon+whisper cannot do: -- On the fly adjustable retention levels. You don't have the space to keep 6 weeks of 1m data? Just reduce it. Or increase it. Our system can do that on the fly. -- Currently in progress: On the fly addition of new retention levels. Have an emergency and need data in greater resolution? Just add a retention level with 1 datapoint / 5s, keep the full data history and tell your data collection to collect more data and delete it later on again wiithout losing data. -- High fault tolerance. We are relying on cassandra for persistent storage, and a properly deployed cassandra cluster with redundancy just doesn't care. Add a new machine, tell everything to rebuild the cluster and the frontend didn't even notice the outage. So, after this wall of text, there are two questions from me: a) is this project interesting enough for everyone? :) b) Are there people who would volunteer to coach me and my team through the proposal and the incubator? Regards, Harald. -- *Harald Krämer* Server Developer (Profiling first) *hkrae...@goodgamestudios.com hkrae...@goodgamestudios.com* Goodgame Studios Theodorstr. 42-90, House 9 22761 Hamburg, Germany Phone: +49 (0)40 219 880 -0 *www.goodgamestudios.com http://www.goodgamestudios.com* Goodgame Studios is a branch of Altigi GmbH Altigi GmbH, District court Hamburg, HRB 99869 Board of directors: Dr. Kai Wawrzinek, Dr. Christian Wawrzinek, Fabian Ritter
Re: Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
Hi Bifroest sounds like a very interesting project and within my field of experience. I have worked about 3/4 year to implement circonus in ASF (It was then decided for good reasons not to use it for alerting), and before that I designed SCADA systems to monitor/control electrical grids. today I live in southern spain (I am danish) so TZ fits nicely. I volunteer to champion for you, if the project want it, but suggest we exchange some mails offlist to checkout your wishes and my possibilities. rgds jan I. On 7 October 2014 10:59, Harald Kraemer hkrae...@goodgamestudios.com wrote: Hi, we have been allowed to open-source one of our company internal projects - currently called Bifroest. Bifroest is a storage backend for graphite-web, based on Apache Cassandra. I'm quite happy about this, and now I'm in the process of finding the best options and means to do so. This mail isn't an entire proposal yet, but I will try to stick at least to the major points in a proposal. What does Bifroest do, and where does it come from. At GoodgameStudios, we used Munin for most of our monitoring, using a lot of custom plugins for our servers and pushing 500 - 700 hosts around. That's ambitious with munin and by now, the munin-master is not able to take the stress anymore. As such, we started to evaluate graphite, since graphite is the state of the art larger scale monitoring solution. To start evaluating graphite, we deployed graphite with a carbon backend on a virtual machine. Our senior monitoring admin (which we didn't have back then) probably just had to giggle a bit and doesn't know why - things didn't perform that well on a virtual machine. It could handle the important data, but the system didn't seem to scale that well. An admin would have tossed hardware at this, SSD-Raids and all that, naturally. But we are software engineers, not admins, thus we tossed software at it (until we required hardware) :) Our intention was to have a graphite with data stored in a distributed database. A distributed database would scale both in storage space and in load the system can deal with. And it's all behind a well-defined interface. That seemed like a nifty feature for a scalable monitoring system. Hence, we tried Cyanide, since Cyanide was just that. Tossed a lot of data into Apache Cassandra, click on the metric tree and... well. Nothing happened, since Cyanide figured that a select * across several 100k rows is a grand idea. After that, we looked at InfluxDB, but at the time we started developing this, InfluxDB didn't support data aggregation and seemed to be in a very, very early stage of development. Thus, the first thought of bifroest was born: Why don't we take the good parts of Cyanide, a solid distributed database, such as Apache Cassandra, and the good parts of carbon and toss them in a big stew? That's what we did, and that's what we are currently deploying as our productive monitoring system, graphite on bifroest as a frontend for apache cassandra. Fun features of this system include: - Existing graphite and most carbon apis: -- Full support of the graphite rest API, since we are just a backend. -- Support for the Plaintext Protocol of Carbon -- Planned: An AMQP interface to handle globally distributed networks - Neat things, which graphite could do as well: -- A fast key cache -- A fast value-cache, which is fed by the data collection to hit the database as little as possible - New things, Graphite+carbon+whisper cannot do: -- On the fly adjustable retention levels. You don't have the space to keep 6 weeks of 1m data? Just reduce it. Or increase it. Our system can do that on the fly. -- Currently in progress: On the fly addition of new retention levels. Have an emergency and need data in greater resolution? Just add a retention level with 1 datapoint / 5s, keep the full data history and tell your data collection to collect more data and delete it later on again wiithout losing data. -- High fault tolerance. We are relying on cassandra for persistent storage, and a properly deployed cassandra cluster with redundancy just doesn't care. Add a new machine, tell everything to rebuild the cluster and the frontend didn't even notice the outage. So, after this wall of text, there are two questions from me: a) is this project interesting enough for everyone? :) b) Are there people who would volunteer to coach me and my team through the proposal and the incubator? Regards, Harald. -- *Harald Krämer* Server Developer (Profiling first) *hkrae...@goodgamestudios.com hkrae...@goodgamestudios.com* Goodgame Studios Theodorstr. 42-90, House 9 22761 Hamburg, Germany Phone: +49 (0)40 219 880 -0 *www.goodgamestudios.com http://www.goodgamestudios.com* Goodgame Studios is a branch of Altigi GmbH Altigi GmbH, District court Hamburg, HRB 99869 Board of directors: Dr. Kai Wawrzinek, Dr. Christian Wawrzinek, Fabian Ritter
Re: Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
Hi Harald, it looks very interesting. Don't hesitate to ping me if you need any help. Regards JB On 10/07/2014 10:59 AM, Harald Kraemer wrote: Hi, we have been allowed to open-source one of our company internal projects - currently called Bifroest. Bifroest is a storage backend for graphite-web, based on Apache Cassandra. I'm quite happy about this, and now I'm in the process of finding the best options and means to do so. This mail isn't an entire proposal yet, but I will try to stick at least to the major points in a proposal. What does Bifroest do, and where does it come from. At GoodgameStudios, we used Munin for most of our monitoring, using a lot of custom plugins for our servers and pushing 500 - 700 hosts around. That's ambitious with munin and by now, the munin-master is not able to take the stress anymore. As such, we started to evaluate graphite, since graphite is the state of the art larger scale monitoring solution. To start evaluating graphite, we deployed graphite with a carbon backend on a virtual machine. Our senior monitoring admin (which we didn't have back then) probably just had to giggle a bit and doesn't know why - things didn't perform that well on a virtual machine. It could handle the important data, but the system didn't seem to scale that well. An admin would have tossed hardware at this, SSD-Raids and all that, naturally. But we are software engineers, not admins, thus we tossed software at it (until we required hardware) :) Our intention was to have a graphite with data stored in a distributed database. A distributed database would scale both in storage space and in load the system can deal with. And it's all behind a well-defined interface. That seemed like a nifty feature for a scalable monitoring system. Hence, we tried Cyanide, since Cyanide was just that. Tossed a lot of data into Apache Cassandra, click on the metric tree and... well. Nothing happened, since Cyanide figured that a select * across several 100k rows is a grand idea. After that, we looked at InfluxDB, but at the time we started developing this, InfluxDB didn't support data aggregation and seemed to be in a very, very early stage of development. Thus, the first thought of bifroest was born: Why don't we take the good parts of Cyanide, a solid distributed database, such as Apache Cassandra, and the good parts of carbon and toss them in a big stew? That's what we did, and that's what we are currently deploying as our productive monitoring system, graphite on bifroest as a frontend for apache cassandra. Fun features of this system include: - Existing graphite and most carbon apis: -- Full support of the graphite rest API, since we are just a backend. -- Support for the Plaintext Protocol of Carbon -- Planned: An AMQP interface to handle globally distributed networks - Neat things, which graphite could do as well: -- A fast key cache -- A fast value-cache, which is fed by the data collection to hit the database as little as possible - New things, Graphite+carbon+whisper cannot do: -- On the fly adjustable retention levels. You don't have the space to keep 6 weeks of 1m data? Just reduce it. Or increase it. Our system can do that on the fly. -- Currently in progress: On the fly addition of new retention levels. Have an emergency and need data in greater resolution? Just add a retention level with 1 datapoint / 5s, keep the full data history and tell your data collection to collect more data and delete it later on again wiithout losing data. -- High fault tolerance. We are relying on cassandra for persistent storage, and a properly deployed cassandra cluster with redundancy just doesn't care. Add a new machine, tell everything to rebuild the cluster and the frontend didn't even notice the outage. So, after this wall of text, there are two questions from me: a) is this project interesting enough for everyone? :) b) Are there people who would volunteer to coach me and my team through the proposal and the incubator? Regards, Harald. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
Harald Kraemer wrote on Tue, Oct 07, 2014 at 10:59:35 +0200: Thus, the first thought of bifroest was born: Why don't we take the good parts of Cyanide, a solid distributed database, such as Apache Cassandra, and the good parts of carbon and toss them in a big stew? That's what we did, and that's what we are currently deploying as our productive monitoring system, graphite on bifroest as a frontend for apache cassandra. ... a) is this project interesting enough for everyone? :) b) Are there people who would volunteer to coach me and my team through the proposal and the incubator? Happy to help. My relevant experience is in the Apache Subversion development community. Daniel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
Hi Harald I have been working on a similar project which enables carbon to have a plugable backend storage system that leverages Apache Cassandra for storage. I opened pull requests in both carbon and graphite for the plugable backend portion and the Cassandra backend is still in the works. Your projects topic is something that I am very familiar with and all the mentioned related technologies. I would be happy to help as either a champion or a mentor for this project -Jake On Tue, Oct 7, 2014 at 4:59 AM, Harald Kraemer hkrae...@goodgamestudios.com wrote: Hi, we have been allowed to open-source one of our company internal projects - currently called Bifroest. Bifroest is a storage backend for graphite-web, based on Apache Cassandra. I'm quite happy about this, and now I'm in the process of finding the best options and means to do so. This mail isn't an entire proposal yet, but I will try to stick at least to the major points in a proposal. What does Bifroest do, and where does it come from. At GoodgameStudios, we used Munin for most of our monitoring, using a lot of custom plugins for our servers and pushing 500 - 700 hosts around. That's ambitious with munin and by now, the munin-master is not able to take the stress anymore. As such, we started to evaluate graphite, since graphite is the state of the art larger scale monitoring solution. To start evaluating graphite, we deployed graphite with a carbon backend on a virtual machine. Our senior monitoring admin (which we didn't have back then) probably just had to giggle a bit and doesn't know why - things didn't perform that well on a virtual machine. It could handle the important data, but the system didn't seem to scale that well. An admin would have tossed hardware at this, SSD-Raids and all that, naturally. But we are software engineers, not admins, thus we tossed software at it (until we required hardware) :) Our intention was to have a graphite with data stored in a distributed database. A distributed database would scale both in storage space and in load the system can deal with. And it's all behind a well-defined interface. That seemed like a nifty feature for a scalable monitoring system. Hence, we tried Cyanide, since Cyanide was just that. Tossed a lot of data into Apache Cassandra, click on the metric tree and... well. Nothing happened, since Cyanide figured that a select * across several 100k rows is a grand idea. After that, we looked at InfluxDB, but at the time we started developing this, InfluxDB didn't support data aggregation and seemed to be in a very, very early stage of development. Thus, the first thought of bifroest was born: Why don't we take the good parts of Cyanide, a solid distributed database, such as Apache Cassandra, and the good parts of carbon and toss them in a big stew? That's what we did, and that's what we are currently deploying as our productive monitoring system, graphite on bifroest as a frontend for apache cassandra. Fun features of this system include: - Existing graphite and most carbon apis: -- Full support of the graphite rest API, since we are just a backend. -- Support for the Plaintext Protocol of Carbon -- Planned: An AMQP interface to handle globally distributed networks - Neat things, which graphite could do as well: -- A fast key cache -- A fast value-cache, which is fed by the data collection to hit the database as little as possible - New things, Graphite+carbon+whisper cannot do: -- On the fly adjustable retention levels. You don't have the space to keep 6 weeks of 1m data? Just reduce it. Or increase it. Our system can do that on the fly. -- Currently in progress: On the fly addition of new retention levels. Have an emergency and need data in greater resolution? Just add a retention level with 1 datapoint / 5s, keep the full data history and tell your data collection to collect more data and delete it later on again wiithout losing data. -- High fault tolerance. We are relying on cassandra for persistent storage, and a properly deployed cassandra cluster with redundancy just doesn't care. Add a new machine, tell everything to rebuild the cluster and the frontend didn't even notice the outage. So, after this wall of text, there are two questions from me: a) is this project interesting enough for everyone? :) b) Are there people who would volunteer to coach me and my team through the proposal and the incubator? Regards, Harald. -- *Harald Krämer* Server Developer (Profiling first) *hkrae...@goodgamestudios.com hkrae...@goodgamestudios.com* Goodgame Studios Theodorstr. 42-90, House 9 22761 Hamburg, Germany Phone: +49 (0)40 219 880 -0 *www.goodgamestudios.com http://www.goodgamestudios.com* Goodgame Studios is a branch of Altigi GmbH Altigi GmbH, District court Hamburg, HRB 99869 Board of directors: Dr. Kai Wawrzinek, Dr. Christian Wawrzinek, Fabian Ritter
Re: Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
Hi, am I looking at the right pull requests with graphite-project/carbon#210 and #216? Quite interesting. Sadly, I don't think I could provide exactly that python API with our existing storage input frontend. Just look out for the issues we fixed in https://github.com/graphite-project/graphite-web/pull/698 :) - Harald 2014-10-07 14:35 GMT+02:00 Jake Farrell jfarr...@apache.org: Hi Harald I have been working on a similar project which enables carbon to have a plugable backend storage system that leverages Apache Cassandra for storage. I opened pull requests in both carbon and graphite for the plugable backend portion and the Cassandra backend is still in the works. Your projects topic is something that I am very familiar with and all the mentioned related technologies. I would be happy to help as either a champion or a mentor for this project -Jake On Tue, Oct 7, 2014 at 4:59 AM, Harald Kraemer hkrae...@goodgamestudios.com wrote: Hi, we have been allowed to open-source one of our company internal projects - currently called Bifroest. Bifroest is a storage backend for graphite-web, based on Apache Cassandra. I'm quite happy about this, and now I'm in the process of finding the best options and means to do so. This mail isn't an entire proposal yet, but I will try to stick at least to the major points in a proposal. What does Bifroest do, and where does it come from. At GoodgameStudios, we used Munin for most of our monitoring, using a lot of custom plugins for our servers and pushing 500 - 700 hosts around. That's ambitious with munin and by now, the munin-master is not able to take the stress anymore. As such, we started to evaluate graphite, since graphite is the state of the art larger scale monitoring solution. To start evaluating graphite, we deployed graphite with a carbon backend on a virtual machine. Our senior monitoring admin (which we didn't have back then) probably just had to giggle a bit and doesn't know why - things didn't perform that well on a virtual machine. It could handle the important data, but the system didn't seem to scale that well. An admin would have tossed hardware at this, SSD-Raids and all that, naturally. But we are software engineers, not admins, thus we tossed software at it (until we required hardware) :) Our intention was to have a graphite with data stored in a distributed database. A distributed database would scale both in storage space and in load the system can deal with. And it's all behind a well-defined interface. That seemed like a nifty feature for a scalable monitoring system. Hence, we tried Cyanide, since Cyanide was just that. Tossed a lot of data into Apache Cassandra, click on the metric tree and... well. Nothing happened, since Cyanide figured that a select * across several 100k rows is a grand idea. After that, we looked at InfluxDB, but at the time we started developing this, InfluxDB didn't support data aggregation and seemed to be in a very, very early stage of development. Thus, the first thought of bifroest was born: Why don't we take the good parts of Cyanide, a solid distributed database, such as Apache Cassandra, and the good parts of carbon and toss them in a big stew? That's what we did, and that's what we are currently deploying as our productive monitoring system, graphite on bifroest as a frontend for apache cassandra. Fun features of this system include: - Existing graphite and most carbon apis: -- Full support of the graphite rest API, since we are just a backend. -- Support for the Plaintext Protocol of Carbon -- Planned: An AMQP interface to handle globally distributed networks - Neat things, which graphite could do as well: -- A fast key cache -- A fast value-cache, which is fed by the data collection to hit the database as little as possible - New things, Graphite+carbon+whisper cannot do: -- On the fly adjustable retention levels. You don't have the space to keep 6 weeks of 1m data? Just reduce it. Or increase it. Our system can do that on the fly. -- Currently in progress: On the fly addition of new retention levels. Have an emergency and need data in greater resolution? Just add a retention level with 1 datapoint / 5s, keep the full data history and tell your data collection to collect more data and delete it later on again wiithout losing data. -- High fault tolerance. We are relying on cassandra for persistent storage, and a properly deployed cassandra cluster with redundancy just doesn't care. Add a new machine, tell everything to rebuild the cluster and the frontend didn't even notice the outage. So, after this wall of text, there are two questions from me: a) is this project interesting enough for everyone? :) b) Are there people who would volunteer to coach me and my team through the proposal and the incubator? Regards,
Re: Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
Hey Harold Those pull requests came after mine, see carbon#139 [1]. Nice catch in 698, in ours we took a different approach and loaded a custom finder within the database plugin [2] itself. Looking forward to comparing notes and helping where I can -Jake [1]: https://github.com/graphite-project/carbon/pull/139 [2]: https://github.com/jfarrell/graphite-web/commit/af90af4d67a2b32d450d4559e06ae2b2fddcc4f1 On Tue, Oct 7, 2014 at 10:20 AM, Harald Kraemer hkrae...@goodgamestudios.com wrote: Hi, am I looking at the right pull requests with graphite-project/carbon#210 and #216? Quite interesting. Sadly, I don't think I could provide exactly that python API with our existing storage input frontend. Just look out for the issues we fixed in https://github.com/graphite-project/graphite-web/pull/698 :) - Harald 2014-10-07 14:35 GMT+02:00 Jake Farrell jfarr...@apache.org: Hi Harald I have been working on a similar project which enables carbon to have a plugable backend storage system that leverages Apache Cassandra for storage. I opened pull requests in both carbon and graphite for the plugable backend portion and the Cassandra backend is still in the works. Your projects topic is something that I am very familiar with and all the mentioned related technologies. I would be happy to help as either a champion or a mentor for this project -Jake On Tue, Oct 7, 2014 at 4:59 AM, Harald Kraemer hkrae...@goodgamestudios.com wrote: Hi, we have been allowed to open-source one of our company internal projects - currently called Bifroest. Bifroest is a storage backend for graphite-web, based on Apache Cassandra. I'm quite happy about this, and now I'm in the process of finding the best options and means to do so. This mail isn't an entire proposal yet, but I will try to stick at least to the major points in a proposal. What does Bifroest do, and where does it come from. At GoodgameStudios, we used Munin for most of our monitoring, using a lot of custom plugins for our servers and pushing 500 - 700 hosts around. That's ambitious with munin and by now, the munin-master is not able to take the stress anymore. As such, we started to evaluate graphite, since graphite is the state of the art larger scale monitoring solution. To start evaluating graphite, we deployed graphite with a carbon backend on a virtual machine. Our senior monitoring admin (which we didn't have back then) probably just had to giggle a bit and doesn't know why - things didn't perform that well on a virtual machine. It could handle the important data, but the system didn't seem to scale that well. An admin would have tossed hardware at this, SSD-Raids and all that, naturally. But we are software engineers, not admins, thus we tossed software at it (until we required hardware) :) Our intention was to have a graphite with data stored in a distributed database. A distributed database would scale both in storage space and in load the system can deal with. And it's all behind a well-defined interface. That seemed like a nifty feature for a scalable monitoring system. Hence, we tried Cyanide, since Cyanide was just that. Tossed a lot of data into Apache Cassandra, click on the metric tree and... well. Nothing happened, since Cyanide figured that a select * across several 100k rows is a grand idea. After that, we looked at InfluxDB, but at the time we started developing this, InfluxDB didn't support data aggregation and seemed to be in a very, very early stage of development. Thus, the first thought of bifroest was born: Why don't we take the good parts of Cyanide, a solid distributed database, such as Apache Cassandra, and the good parts of carbon and toss them in a big stew? That's what we did, and that's what we are currently deploying as our productive monitoring system, graphite on bifroest as a frontend for apache cassandra. Fun features of this system include: - Existing graphite and most carbon apis: -- Full support of the graphite rest API, since we are just a backend. -- Support for the Plaintext Protocol of Carbon -- Planned: An AMQP interface to handle globally distributed networks - Neat things, which graphite could do as well: -- A fast key cache -- A fast value-cache, which is fed by the data collection to hit the database as little as possible - New things, Graphite+carbon+whisper cannot do: -- On the fly adjustable retention levels. You don't have the space to keep 6 weeks of 1m data? Just reduce it. Or increase it. Our system can do that on the fly. -- Currently in progress: On the fly addition of new retention levels. Have an emergency and need data in greater resolution? Just add a retention level with 1 datapoint / 5s, keep the full data history and tell your data collection to collect more data and delete it later on again wiithout losing data. --
Re: [VOTE] Apache Drill 0.6.0-incubating release
Hi. I have had a look at your release and it looks good, I could not find any formal errors. But I took a closer look at the release vote thread, because a failing unit test is a serious bad quality signal for me. Whenever I test new software, I download it, build it, then run all test cases to secure I got it build correctly, where I assume I have made something wrong if a test case fails. Apache is known for quality software and I think we all want to keep that image. I am sure the project does not take quality lightly, but the attitude can be fixed later especially with unit tests is to me not a good policy. If the software only runs with 1.7 and not higher, then why not make a simple startup version check, then there would be no problem (of course its even better to solve the problem). I just wonder how this error will affect people using the project. It seems (from the vote thread) you already have solved the problem, but dont want to wait for a respin, can you please at least explain why the project is under such a time constraint, that 72 hours is too long to wait to make good quality. I wanted to give the release a -1 but decided to give -0 binding. in the hope the PMC will go for quality and voluntary respin the release. rgds jan i On 7 October 2014 07:09, Steven Phillips sphill...@maprtech.com wrote: In case there is any confusion, the first email sent out in this thread had the wrong vote count. The second one has the correct count: +9 binding +3 non-binding I should also mention there was one -0 (binding). This was due to unit test failures when using java 1.8. Jiras were filed, and the fix will be included in the next release, not this one. On Sun, Oct 5, 2014 at 11:14 AM, Steven Phillips s...@apache.org wrote: I would like to present the Apache Drill 0.6.0-incubating release to the general incubator list for a vote. This set of artifacts have passed our drill-dev vote and incorporate a number of improvements with over 30 JIRAs closed in the last month. The vote thread can be found here: http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201410.mbox/%3CCAA_-67fAJFB20wGX462wm7BYvoSy3PvydCPgY9uNSEj3HpQRmg%40mail.gmail.com%3E The vote passed with: +9 binding +3 non-binding You can find the artifacts for the release at this location: http://people.apache.org/~smp/apache-drill-0.6.0.rc0/ I look forward to your feedback. Thanks, Steven -- Steven Phillips Software Engineer mapr.com
Re: [VOTE] Apache Drill 0.6.0-incubating release
The unit test was only on Java 1.8. That problem will be resolved in the next release which will be in roughly a month from now. The current primary target of Drill is 1.7. The number of reviewers for the release is an indication of how the community doesn't view Java 1.8 as a critical platform at this time. On Tue, Oct 7, 2014 at 8:41 AM, jan i j...@apache.org wrote: Hi. I have had a look at your release and it looks good, I could not find any formal errors. But I took a closer look at the release vote thread, because a failing unit test is a serious bad quality signal for me. Whenever I test new software, I download it, build it, then run all test cases to secure I got it build correctly, where I assume I have made something wrong if a test case fails. Apache is known for quality software and I think we all want to keep that image. I am sure the project does not take quality lightly, but the attitude can be fixed later especially with unit tests is to me not a good policy. If the software only runs with 1.7 and not higher, then why not make a simple startup version check, then there would be no problem (of course its even better to solve the problem). I just wonder how this error will affect people using the project. It seems (from the vote thread) you already have solved the problem, but dont want to wait for a respin, can you please at least explain why the project is under such a time constraint, that 72 hours is too long to wait to make good quality. I wanted to give the release a -1 but decided to give -0 binding. in the hope the PMC will go for quality and voluntary respin the release. rgds jan i On 7 October 2014 07:09, Steven Phillips sphill...@maprtech.com wrote: In case there is any confusion, the first email sent out in this thread had the wrong vote count. The second one has the correct count: +9 binding +3 non-binding I should also mention there was one -0 (binding). This was due to unit test failures when using java 1.8. Jiras were filed, and the fix will be included in the next release, not this one. On Sun, Oct 5, 2014 at 11:14 AM, Steven Phillips s...@apache.org wrote: I would like to present the Apache Drill 0.6.0-incubating release to the general incubator list for a vote. This set of artifacts have passed our drill-dev vote and incorporate a number of improvements with over 30 JIRAs closed in the last month. The vote thread can be found here: http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201410.mbox/%3CCAA_-67fAJFB20wGX462wm7BYvoSy3PvydCPgY9uNSEj3HpQRmg%40mail.gmail.com%3E The vote passed with: +9 binding +3 non-binding You can find the artifacts for the release at this location: http://people.apache.org/~smp/apache-drill-0.6.0.rc0/ I look forward to your feedback. Thanks, Steven -- Steven Phillips Software Engineer mapr.com
Re: [VOTE] Apache Drill 0.6.0-incubating release
On Tue, Oct 7, 2014 at 8:41 AM, jan i j...@apache.org wrote: It seems (from the vote thread) you already have solved the problem, but dont want to wait for a respin, can you please at least explain why the project is under such a time constraint, that 72 hours is too long to wait to make good quality. You have to make a cut somewhere. There are always a number of very small, low priority bugs and platform issues in any release. Drill is now on a monthly release cycle so triaging a minor bug as something that doesn't stop shipping has very little down-side. On the other hand, delaying by 3+ days is more than a 10% delay in the monthly release. Your characterization of this issue being a quality issue is also a bit of an over-statement. The problem was that several dependencies worked differently on Java8 than Java7. The fix involved changing the version of these dependencies. Changing dependency versions is not a small change and requires a full QA cycle since it takes serious thought to decide what impact the version change might have. The best way for that to happen in a reasonable way is to simply put this fix in the next release.
Re: [VOTE] Apache Drill 0.6.0-incubating release
Hi Jan, The issue was discussed on the release voting thread and there seems to be an agreement in the community that it may not be worth holding the release to include Java 8 support since 1.) Among most of the users, as evident on the vote thread, very few are running Java 8 in dev and test environment and even fewer are running it in production. 2.) The changes that seem to fix the test failures involves moving to a very new release of few libraries which are tightly integrated with Drill's core functionality and hence we would like to test these a bit more before merging in to a release. 3.) With our goal to have shorter and more frequent (monthly) releases, we try to be little judicious with picking what issue could have a wider user impact and should be fixed immediately, vs something which affects a very small percentage of use cases. 4.) And lastly, as Julian mentioned on the thread that the set of fixes might not be complete yet, I think we need more time before we can merge these changes in to a release with confidence to support a new platform. I hope this persuades you to reconsider your position on this release candidate. Regards, aditya... On Tue, Oct 7, 2014 at 12:02 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Tue, Oct 7, 2014 at 8:41 AM, jan i j...@apache.org wrote: It seems (from the vote thread) you already have solved the problem, but dont want to wait for a respin, can you please at least explain why the project is under such a time constraint, that 72 hours is too long to wait to make good quality. You have to make a cut somewhere. There are always a number of very small, low priority bugs and platform issues in any release. Drill is now on a monthly release cycle so triaging a minor bug as something that doesn't stop shipping has very little down-side. On the other hand, delaying by 3+ days is more than a 10% delay in the monthly release. Your characterization of this issue being a quality issue is also a bit of an over-statement. The problem was that several dependencies worked differently on Java8 than Java7. The fix involved changing the version of these dependencies. Changing dependency versions is not a small change and requires a full QA cycle since it takes serious thought to decide what impact the version change might have. The best way for that to happen in a reasonable way is to simply put this fix in the next release.
Re: [VOTE] Accept Lens into the Apache Incubator (earlier called Grill)
On Mon, Oct 6, 2014 at 4:51 AM, Sharad Agarwal sha...@apache.org wrote: Following the discussion earlier in the thread https://www.mail-archive.com/general@incubator.apache.org/msg45208.html I would like to call a Vote for accepting Lens as a new incubator project. The proposal is available at: https://wiki.apache.org/incubator/LensProposal Vote is open till Oct 09, 2014 4 PM PST. [ ] +1 accept Lens in the Incubator [ ] +/-0 [ ] -1 because... +1 (binding) Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Seeking interest and a champion for bifroest - a backend for graphite-web, on Apache Cassandra
On Tue, Oct 7, 2014 at 1:59 AM, Harald Kraemer hkrae...@goodgamestudios.com wrote: So, after this wall of text, there are two questions from me: a) is this project interesting enough for everyone? :) b) Are there people who would volunteer to coach me and my team through the proposal and the incubator? This sounds really interesting! The only thing I'd strongly encourage you to do is to get feedback from Apache Cassandra community. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Apache Drill 0.6.0-incubating release
Hi First let me make it clear I am not a java specialist, and secondly big thanks to both of you for explaining more in detail what the problem is. On 7 October 2014 21:24, Aditya a...@apache.org wrote: Hi Jan, The issue was discussed on the release voting thread and there seems to be an agreement in the community that it may not be worth holding the release to include Java 8 support since 1.) Among most of the users, as evident on the vote thread, very few are running Java 8 in dev and test environment and even fewer are running it in production. 2.) The changes that seem to fix the test failures involves moving to a very new release of few libraries which are tightly integrated with Drill's core functionality and hence we would like to test these a bit more before merging in to a release. very fair I would have done the same. 3.) With our goal to have shorter and more frequent (monthly) releases, we try to be little judicious with picking what issue could have a wider user impact and should be fixed immediately, vs something which affects a very small percentage of use cases. I agree with you in principle, but when the project have bothered to write a unit-test case, it means its an important functionality. 4.) And lastly, as Julian mentioned on the thread that the set of fixes might not be complete yet, I think we need more time before we can merge these changes in to a release with confidence to support a new platform. I agree with this, but it still does not explain why the software not simply deny runing with java1.7+. To me it seems a simple if during load could solve the problem, and is in general something a software should do with all dependencies especially when newer versions dont work (most of us expect newer versions to be backward compatible). I hope this persuades you to reconsider your position on this release candidate. I hope you noted, that I just raised a concern and did not block the release. Regards, aditya... On Tue, Oct 7, 2014 at 12:02 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Tue, Oct 7, 2014 at 8:41 AM, jan i j...@apache.org wrote: It seems (from the vote thread) you already have solved the problem, but dont want to wait for a respin, can you please at least explain why the project is under such a time constraint, that 72 hours is too long to wait to make good quality. You have to make a cut somewhere. There are always a number of very small, low priority bugs and platform issues in any release. Drill is now on a monthly release cycle so triaging a minor bug as something that doesn't stop shipping has very little down-side. I actually agree with the above statement. A release will contain a number of open issues, or errors yet to be found. But to me a failing unit test, is not a low priority bug because it shows that the project feel the tested functionality is important and if the unit test fails, the software will fail for some users. On the other hand, delaying by 3+ days is more than a 10% delay in the monthly release. Is there any demand that makes it nessecary to make an exact monthly releaseor can it be 3weeks sometimes and 5weeks other times ? Your characterization of this issue being a quality issue is also a bit of an over-statement. The problem was that several dependencies worked differently on Java8 than Java7. The fix involved changing the version of these dependencies. Changing dependency versions is not a small change and requires a full QA cycle since it takes serious thought to decide what impact the version change might have. The best way for that to happen in a reasonable way is to simply put this fix in the next release. My intentation was to raise a concern, NOT to block the release (I did on purpose give a 0 and not -1). I am sorry if you feel its an over-statement, but honestly for me failing unit tests and not checking a version dependency (in case of older versions) is cause for alarm. Java1.8 is not exactly brand new, so if the project depend on version 1.7, I really expect the software to check the installed version, and not simply blindly trust the users have installed whats required. In general when I read release notes stating a version, I assume that never versions are ok. I do agree that upgrading to a newer version requires more time and far more testing. I think the project is doing a fine job and look forward to give many +1 in the future, so please dont be upset that I point out something which I feel is important, especially since I do it in a politle way without blocking anything. rgds jan i.
Re: [VOTE] Apache Drill 0.6.0-incubating release
Jan, Your concern was a valid one and definitely one worth explanation. Please do not take my response as a rant from an upset soul :), rather an explanation towards why we chose to do what we did. I apologize if I sounded like one. We are, sometime, limited by the tools that we use and one such limitation of maven's unit test framework which Drill uses is that there is no clean way to enforce upper bound on Java version during test execution. And in the spirit of exploration and tinkering, among the core tenets of open source philosophy, I do not think we should block anyone from trying something unknown, a new platform in this case. We do though, test for and indicate for the known incompatibility. However, I do agree that a well defined list of supported platform helps minimize confusion and makes the users' life a tad easier. To that effect, we will keep the list up-to-date. We thank you and look forward to your participation in our discussions in future. Regards, aditya... On Tue, Oct 7, 2014 at 2:46 PM, jan i j...@apache.org wrote: Hi First let me make it clear I am not a java specialist, and secondly big thanks to both of you for explaining more in detail what the problem is. On 7 October 2014 21:24, Aditya a...@apache.org wrote: Hi Jan, The issue was discussed on the release voting thread and there seems to be an agreement in the community that it may not be worth holding the release to include Java 8 support since 1.) Among most of the users, as evident on the vote thread, very few are running Java 8 in dev and test environment and even fewer are running it in production. 2.) The changes that seem to fix the test failures involves moving to a very new release of few libraries which are tightly integrated with Drill's core functionality and hence we would like to test these a bit more before merging in to a release. very fair I would have done the same. 3.) With our goal to have shorter and more frequent (monthly) releases, we try to be little judicious with picking what issue could have a wider user impact and should be fixed immediately, vs something which affects a very small percentage of use cases. I agree with you in principle, but when the project have bothered to write a unit-test case, it means its an important functionality. 4.) And lastly, as Julian mentioned on the thread that the set of fixes might not be complete yet, I think we need more time before we can merge these changes in to a release with confidence to support a new platform. I agree with this, but it still does not explain why the software not simply deny runing with java1.7+. To me it seems a simple if during load could solve the problem, and is in general something a software should do with all dependencies especially when newer versions dont work (most of us expect newer versions to be backward compatible). I hope this persuades you to reconsider your position on this release candidate. I hope you noted, that I just raised a concern and did not block the release. Regards, aditya... On Tue, Oct 7, 2014 at 12:02 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Tue, Oct 7, 2014 at 8:41 AM, jan i j...@apache.org wrote: It seems (from the vote thread) you already have solved the problem, but dont want to wait for a respin, can you please at least explain why the project is under such a time constraint, that 72 hours is too long to wait to make good quality. You have to make a cut somewhere. There are always a number of very small, low priority bugs and platform issues in any release. Drill is now on a monthly release cycle so triaging a minor bug as something that doesn't stop shipping has very little down-side. I actually agree with the above statement. A release will contain a number of open issues, or errors yet to be found. But to me a failing unit test, is not a low priority bug because it shows that the project feel the tested functionality is important and if the unit test fails, the software will fail for some users. On the other hand, delaying by 3+ days is more than a 10% delay in the monthly release. Is there any demand that makes it nessecary to make an exact monthly releaseor can it be 3weeks sometimes and 5weeks other times ? Your characterization of this issue being a quality issue is also a bit of an over-statement. The problem was that several dependencies worked differently on Java8 than Java7. The fix involved changing the version of these dependencies. Changing dependency versions is not a small change and requires a full QA cycle since it takes serious thought to decide what impact the version change might have. The best way for that to happen in a reasonable way is to simply put this fix in the next release. My intentation was to raise a concern, NOT
Re: [VOTE] Apache Drill 0.6.0-incubating release
On Tue, Oct 7, 2014 at 2:46 PM, jan i j...@apache.org wrote: 4.) And lastly, as Julian mentioned on the thread that the set of fixes might not be complete yet, I think we need more time before we can merge these changes in to a release with confidence to support a new platform. I agree with this, but it still does not explain why the software not simply deny runing with java1.7+. To me it seems a simple if during load could solve the problem, and is in general something a software should do with all dependencies especially when newer versions dont work (most of us expect newer versions to be backward compatible). That is a plausible action. That would not have changed the fact of the failure under 1.8, it would merely have changed the error message. Until this issue was encountered, however, it was the assumption that the code would run on java8, but it was not considered important enough to specifically test since, as Aditya mentioned, java8 has essentially no production presence so far. As such, testing for java8 and refusing to start would simply guaranteeing failure as opposed to being open to success.
Re: October reporting
On Sun, Oct 5, 2014 at 5:35 PM, Alan D. Cabrera l...@toolazydogs.com wrote: I'm happy to do it again. Thanks!!! Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Apache Drill 0.6.0-incubating release
On Tue, Oct 7, 2014 at 2:46 PM, jan i j...@apache.org wrote: My intentation was to raise a concern, NOT to block the release (I did on purpose give a 0 and not -1). I am sorry if you feel its an over-statement, but honestly for me failing unit tests and not checking a version dependency (in case of older versions) is cause for alarm. The response that you are getting is a measure of how seriously the Drill project takes community building and consensus.
Re: [VOTE] Accept Lens into the Apache Incubator (earlier called Grill)
+1 (non-binding) On Mon, Oct 6, 2014 at 5:21 PM, Sharad Agarwal sha...@apache.org wrote: Following the discussion earlier in the thread https://www.mail-archive.com/general@incubator.apache.org/msg45208.html I would like to call a Vote for accepting Lens as a new incubator project. The proposal is available at: https://wiki.apache.org/incubator/LensProposal Vote is open till Oct 09, 2014 4 PM PST. [ ] +1 accept Lens in the Incubator [ ] +/-0 [ ] -1 because... Only Votes from Incubator PMC members are binding, but all are welcome to express their thoughts. I am +1 (non-binding). Thanks Sharad -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
