RE: Accumulo incubator proposal: Statement of Concern
No... Knowing Noel, he was not opening anything. He was speaking to general principles about how the Incubator works. +1 --- Noel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Doug, As Greg and others have pointed out, none of this really matters, but since, you did ask for some examples, I can think of two immediately: 1) As Dim's pointed out, when CXF entered the incubator, it obviously conflicted directly with Axis2. Many ideas were the same. Many of the goals were the same. Etc... The community was different. There WERE forks of some Apache WebServices things into CXF (a fork of Neethi being the primary one).In the long run, things have worked out quite well with both Axis2 and CXF thriving despite both being written in Java and with likely a 90% overlap in target problems they are trying to solve. The communities have started working together on some of the underlying libs (example: ideas from the fork of Neethi in CXF was pushed into ws to become Neethi 3.0 which they now both use, but it took 4 years to get there), but the bulk of the communities are separate. 2) Wink - when Wink entered the incubator, one of the proposed blocks of code in the grant was a direct fork (with many changes) of the CXF JAX-RS implementation. They ended up going a different direction, but one of the original code bases was a direct fork of another Apache project. They've figured out their niche, the CXF JAX-RS implementation has figure out it's place and both are used. So basically, as a bunch of folks have been saying, in many cases, a fork or a new community duplicating some ideas in a new (or even the same) way is NOT a bad thing. The incubator is here to foster new communities and help them succeed and find their own place in the Apache ecosystem. If that competes/overlaps with another Apache project, SO WHAT. Each community can find their own niche and own solutions. Users can go to whichever solves their problems better. Anyway, you asked for examples, there are the two that popped into my head immediately. Dan On Monday, September 12, 2011 10:34:53 PM Doug Meil wrote: Hi there- I thought the email chain prompted by my questions had a gracious and productive ending several days ago, but if you would like to start this up again, ok. re: turf is being infringed It's funny that you said that, because according to the other emails on Accumulo in the past week there are a couple thousand lines of HBase code in Accumulo, and core functionality at that (e.g., block cache). Even Incubator supporters of Accumulo would admit that's a little unusual in the ASF, and I believe they've said so in other threads I've seen. As was pointed out, HBase copied the Hadoop RPC for it's own purpose, but HBase wasn't trying to be another RPC framework - it was trying to be Hbase - something Hadoop was not. I'm not making a legal argument here because the ASF code license is free beer/speech, etc. It's just that Accumulo is, in a very real sense, actually standing on a part of HBase. If there are other examples of this in Apache I'd be curious to know what they are. That's not a sarcastic request... seriously, I'd like to know because it would be worth documenting as case studies for other projects. re: overlap This is the there are multiple webservers in ASF counter-argument - except that ones that exist are in two different languages (Apache WS and Tomcat). The point I made in my email is that if a 3rd webserver showed up that was written in Java that copied a couple thousand lines of code from Tomcat, I would hope somebody would ask why. I thought I had some agreements from some folks on this thread, but you feel differently so we're going to have to disagree. Thanks. Doug On 9/12/11 8:56 PM, Noel J. Bergman n...@devtech.com wrote: Doug Meil wrote: I think that the ASF and ASF incubator leadership should consider it a priority to foster such communication. We do. I, in particular, tend to do it --- and have on occassion been criticized for trying to foster project collaboration and/or merger. BUT ... The Incubator does not pick winners, exclude projects based on overlap with others, or deny a community a chance just because some other project (ASF or otherwise) feels that their turf is being infringed. We have been very consistent on this issue. Your best move is to open talks with the Accumulo community, and see what options for collaboration and/or merger exist as it proceeds. As for some of the other comments, Accumulo will be held to the same standards as every other project, and will be required to exhibit The Apache Way. --- Noel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail:
Re: Accumulo incubator proposal: Statement of Concern
On Tue, Sep 13, 2011 at 04:34, Doug Meil doug.m...@explorysmedical.com wrote: re: overlap This is the there are multiple webservers in ASF counter-argument - except that ones that exist are in two different languages (Apache WS and Tomcat). The point I made in my email is that if a 3rd webserver showed up that was written in Java that copied a couple thousand lines of code from Tomcat, I would hope somebody would ask why. I thought I had some agreements from some folks on this thread, but you feel differently so we're going to have to disagree. This is not about feeling differently. There is no stealing of code or intellectual property, so as far as incubation is concerned this is a non-argument. Another example: Chukwa, Flume and Ambari. The're all incubating, with significant overlap in features and technical architecture. So what? If successful, some or all of them may become TLPs. I didn't notice similar turf-related complaints from those projects. Bernd - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
RE: Accumulo incubator proposal: Statement of Concern
Doug Meil wrote: I think that the ASF and ASF incubator leadership should consider it a priority to foster such communication. We do. I, in particular, tend to do it --- and have on occassion been criticized for trying to foster project collaboration and/or merger. BUT ... The Incubator does not pick winners, exclude projects based on overlap with others, or deny a community a chance just because some other project (ASF or otherwise) feels that their turf is being infringed. We have been very consistent on this issue. Your best move is to open talks with the Accumulo community, and see what options for collaboration and/or merger exist as it proceeds. As for some of the other comments, Accumulo will be held to the same standards as every other project, and will be required to exhibit The Apache Way. --- Noel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Hi there- I thought the email chain prompted by my questions had a gracious and productive ending several days ago, but if you would like to start this up again, ok. re: turf is being infringed It's funny that you said that, because according to the other emails on Accumulo in the past week there are a couple thousand lines of HBase code in Accumulo, and core functionality at that (e.g., block cache). Even Incubator supporters of Accumulo would admit that's a little unusual in the ASF, and I believe they've said so in other threads I've seen. As was pointed out, HBase copied the Hadoop RPC for it's own purpose, but HBase wasn't trying to be another RPC framework - it was trying to be Hbase - something Hadoop was not. I'm not making a legal argument here because the ASF code license is free beer/speech, etc. It's just that Accumulo is, in a very real sense, actually standing on a part of HBase. If there are other examples of this in Apache I'd be curious to know what they are. That's not a sarcastic request... seriously, I'd like to know because it would be worth documenting as case studies for other projects. re: overlap This is the there are multiple webservers in ASF counter-argument - except that ones that exist are in two different languages (Apache WS and Tomcat). The point I made in my email is that if a 3rd webserver showed up that was written in Java that copied a couple thousand lines of code from Tomcat, I would hope somebody would ask why. I thought I had some agreements from some folks on this thread, but you feel differently so we're going to have to disagree. Thanks. Doug On 9/12/11 8:56 PM, Noel J. Bergman n...@devtech.com wrote: Doug Meil wrote: I think that the ASF and ASF incubator leadership should consider it a priority to foster such communication. We do. I, in particular, tend to do it --- and have on occassion been criticized for trying to foster project collaboration and/or merger. BUT ... The Incubator does not pick winners, exclude projects based on overlap with others, or deny a community a chance just because some other project (ASF or otherwise) feels that their turf is being infringed. We have been very consistent on this issue. Your best move is to open talks with the Accumulo community, and see what options for collaboration and/or merger exist as it proceeds. As for some of the other comments, Accumulo will be held to the same standards as every other project, and will be required to exhibit The Apache Way. --- Noel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Mon, Sep 12, 2011 at 22:34, Doug Meil doug.m...@explorysmedical.com wrote: Hi there- I thought the email chain prompted by my questions had a gracious and productive ending several days ago, but if you would like to start this up again, ok. No... Knowing Noel, he was not opening anything. He was speaking to general principles about how the Incubator works. He just came to the discussion late, and wanted to clarify the Incubator principles. re: turf is being infringed It's funny that you said that, because according to the other emails on Accumulo in the past week there are a couple thousand lines of HBase code in Accumulo, and core functionality at that (e.g., block cache). Even Incubator supporters of Accumulo would admit that's a little unusual in the ASF, and I believe they've said so in other threads I've seen. What is unusual? Building upon other open source projects? Hardly. As was pointed out, HBase copied the Hadoop RPC for it's own purpose, but HBase wasn't trying to be another RPC framework - it was trying to be Hbase - something Hadoop was not. I'm not making a legal argument here because the ASF code license is free beer/speech, etc. It's just that Accumulo is, in a very real sense, actually standing on a part of HBase. So f'ing what?! The ASF releases its code for others to use and to build upon. They are doing *exactly* what we want them to? What is your issue with that? If there are other examples of this in Apache I'd be curious to know what they are. That's not a sarcastic request... seriously, I'd like to know because it would be worth documenting as case studies for other projects. Who cares? Apache produces code to be *used*. Period. If other projects at Apache use it, then awesome. If other projects fork it and change it, then totally fine. It might be nice to feed their changes back, but maybe they have a different goal, and need to take it in a direction the original project will not sign up for. In short: every comment of yours absolutely signals turf war to me. You seemingly continue to take affront at Accumulo for one reason or another, and attempt to justify that position through various hand-waving that flies in the face of what the ASF truly represents. And I say: that stinks. Accumulo has every right, and every encouragement, to do exactly what they're doing. And they are now able to share their work with us. AWESOME. Nothing more difficult than that. re: overlap This is the there are multiple webservers in ASF counter-argument - except that ones that exist are in two different languages (Apache WS and Tomcat). The point I made in my email is that if a 3rd webserver showed up that was written in Java that copied a couple thousand lines of code from Tomcat, I would hope somebody would ask why. I thought I had some agreements from some folks on this thread, but you feel differently so we're going to have to disagree. Language difference was just an example. You're focusing on the wrong thing. The point is that a community desires to accomplish a task in a different way. Whether that is through language, approach, or just a different config file format... that is fine. I'm with Alex on this one: we're here to foster communities. When somebody has a new idea, then we are here to support them. It is not our purpose to shut down communities. -g - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Wed, Sep 7, 2011 at 3:27 AM, Benson Margulies bimargul...@gmail.com wrote: Doug Meil, SNIP Lots of incubator proposals come from groups of people with little or no prior track record at Apache or in Open Source at all. The point of incubation is to give them a chance to either learn how to be an Apache TLP -- or not. We do not tell people 'go away, you haven't enough pedigree.' Mentors have more work to do when the nucleus of a podling is comparatively inexperienced. Well put! They have some community skills to develop but that does not mean they don't deserve a chance. Although I snipped the paragraph you included regarding the nature of government institutions, I think this is a great opportunity for the Apache Way to start infusing into these kinds of organization. Even if the polling terminates before graduation there's a lot to be learned from the effort. Further, I don't know of any policy of the ASF in general, or the incubator in particular, that insists that a group of itch-scratchers make any particular attempt to make any particular gesture towards any particular existing community as a prerequisite of incubation. Five of my friends and I could turn up here tomorrow with a proposal to fork an existing TLP to scratch our particular itch, and, so far as I can tell, the incubator PMC would evaluate the proposal on its merits, independent of the fork. Yep, the point is community, over code. If Project X has a dominant group (say all from the same company) suppressing valid technical advances other than those in line with their agenda, then the suffering minority can choose to fork and propose a new community for incubation. Project X and Project Y can co-exist at Apache. Regardless of our project nomenclature, I don't see incubator proposals as proposals for projects but rather proposals for new communities. The substrate bringing the community together is the code. Forking code bases and incubating parallel communities is not a bad thing: it provides a means to release pressure. I've seen some negative corporate driven dynamics in various new projects. I've made some light comments to improve productivity without any kind of aggressive push yet have been confronted with a negative, we don't do it that way, we've done it this way, our way, for years. I'm appalled by the code quality at the same time but that's a different matter and I'm not going to reopen that extremely long thread we had in the past. If those being pushed away continue to be pushed away then the right to fork and start a new community presents an excellent opportunity for the oppressed voices. Everybody deserves a chance. AFAIC I don't care of this proposal is a complete fork of HBase itself. If they want to build a community in line with the Apache Way then we welcome them to try. If they fail to meet the requirements then that's no skin off our back. You are of course entitled to your opinions and concerns, but if you think that there is some policy that supports the position that PMC members should vote -1 on this proposal due to these issues I wish you would cite it. Ditto. SNIP ... Best Regards, -- Alex - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Doug, I was criticizing my message, not yours in that respect. which should have been clear from the comment justifying. On Sep 6, 2011, at 10:07 PM, Doug Meil doug.m...@explorysmedical.com wrote: Greetings Benson, re: top-posting is generally less than ideal It's funny that you would say that, because there was a 17 message email-chain about Accumulo this past weekend that you participated in, and I noticed that you didn't tell any of those people that posting on this dist-list was a bad idea. http://mail-archives.apache.org/mod_mbox/incubator-general/201109.mbox/brow ser re: pedigree Pedigree is hardly what anybody is looking for. I have been answering questions on the HBase dist-list for 2 years (yes, since HBase 0.20) and I can tell you personally that some of the people on the dist-list are brilliant, and others I wonder how they managed to turn their computer on (hey, did I just say that out loud?). But we try to answer all their questions just the same. I'll let the other HBase team members speak for themselves, but my request to ASF is that if somebody proposes something that nearly exactly like an existing ASF project, and it's implemented in the same language, and it even copies their code - shouldn't somebody at ASF ask the proposing team: have you guys talked to the other project? Especially since, in this case, one of their claims is that HBase can't do or has no plans for something. It's all about community, isn't it? On 9/6/11 8:27 PM, Benson Margulies bimargul...@gmail.com wrote: Doug Meil, Top-posting is generally less than idea, but I want to respond to the overall theme of your message, not to individual points and sentences. The attitude of various chunks of the US government to Open Source has changed radically over the last few years. What would have been institutionally impossible is now encouraged. It is unfair to criticize these people for being under the radar when their management told them to be under the radar and claim that the leopard cannot change its spots. That situation has changed. The code base exists, it serves a particular collection of needs, and here they are. Lots of incubator proposals come from groups of people with little or no prior track record at Apache or in Open Source at all. The point of incubation is to give them a chance to either learn how to be an Apache TLP -- or not. We do not tell people 'go away, you haven't enough pedigree.' Mentors have more work to do when the nucleus of a podling is comparatively inexperienced. Further, I don't know of any policy of the ASF in general, or the incubator in particular, that insists that a group of itch-scratchers make any particular attempt to make any particular gesture towards any particular existing community as a prerequisite of incubation. Five of my friends and I could turn up here tomorrow with a proposal to fork an existing TLP to scratch our particular itch, and, so far as I can tell, the incubator PMC would evaluate the proposal on its merits, independent of the fork. You are of course entitled to your opinions and concerns, but if you think that there is some policy that supports the position that PMC members should vote -1 on this proposal due to these issues I wish you would cite it. In the past, committers of existing TLP's have been, well, particularly vigilant in monitoring the progress of 'competing' podlings. CXF was an example podling of this effect. You are, of course, welcome to adopt that stance. You are also welcome to slurp up every line of their code, once in place, if you think that HBase could benefit from it. Perhaps needless to say, but as a Mentor-in-waiting, I'm all in favor of this proposal. --benson margulies - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On 9/6/2011 10:07 PM, Doug Meil wrote: ...snip... I'll let the other HBase team members speak for themselves, but my request to ASF is that if somebody proposes something that nearly exactly like an existing ASF project, and it's implemented in the same language, and it even copies their code - shouldn't somebody at ASF ask the proposing team: have you guys talked to the other project? Especially since, in this case, one of their claims is that HBase can't do or has no plans for something. It's all about community, isn't it? Yes, a [PROPOSAL] thread is certainly a place to ask about what other Apache projects and communities are related to the proposal, and how they've interacted before. It seems But we're happy to incubate any promising community, even if their technology competes with an existing Apache project. It indeed is about community - and if there's another promising community that wants to fork Subversion and build BigBrother, then we should let them try it. - Shane - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On 9/6/2011 9:56 PM, Doug Meil wrote: Re: Highlander Greatest movie ever. Don't forget the sequel! Highlander: There Should Have Been Only One But in terms of incubator proposals, technical competition is fine. - Shane - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
+1 to that. Holy smokes, that second one really sucked. :-) On 9/7/11 8:55 AM, Shane Curcuru a...@shanecurcuru.org wrote: On 9/6/2011 9:56 PM, Doug Meil wrote: Re: Highlander Greatest movie ever. Don't forget the sequel! Highlander: There Should Have Been Only One But in terms of incubator proposals, technical competition is fine. - Shane - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Thanks for the feedback folks. I would caution that it is not possible to simultaneously optimize for both of these goals... 1) Community over code 2) Competition is good ... as both will be lost. Particularly where the latter contains the absence of inter-project communication, and competition-at-any-price. Without communication, there is no community. I think that the ASF and ASF incubator leadership should consider it a priority to foster such communication. And this means asking the questions I asked before, e.g,. you guys are trying to do the same thing as... Have you talked them? Consider it Open-Source Project Parenting. Doug On 9/7/11 3:16 AM, Alex Karasulu akaras...@apache.org wrote: On Wed, Sep 7, 2011 at 3:27 AM, Benson Margulies bimargul...@gmail.com wrote: Doug Meil, SNIP Lots of incubator proposals come from groups of people with little or no prior track record at Apache or in Open Source at all. The point of incubation is to give them a chance to either learn how to be an Apache TLP -- or not. We do not tell people 'go away, you haven't enough pedigree.' Mentors have more work to do when the nucleus of a podling is comparatively inexperienced. Well put! They have some community skills to develop but that does not mean they don't deserve a chance. Although I snipped the paragraph you included regarding the nature of government institutions, I think this is a great opportunity for the Apache Way to start infusing into these kinds of organization. Even if the polling terminates before graduation there's a lot to be learned from the effort. Further, I don't know of any policy of the ASF in general, or the incubator in particular, that insists that a group of itch-scratchers make any particular attempt to make any particular gesture towards any particular existing community as a prerequisite of incubation. Five of my friends and I could turn up here tomorrow with a proposal to fork an existing TLP to scratch our particular itch, and, so far as I can tell, the incubator PMC would evaluate the proposal on its merits, independent of the fork. Yep, the point is community, over code. If Project X has a dominant group (say all from the same company) suppressing valid technical advances other than those in line with their agenda, then the suffering minority can choose to fork and propose a new community for incubation. Project X and Project Y can co-exist at Apache. Regardless of our project nomenclature, I don't see incubator proposals as proposals for projects but rather proposals for new communities. The substrate bringing the community together is the code. Forking code bases and incubating parallel communities is not a bad thing: it provides a means to release pressure. I've seen some negative corporate driven dynamics in various new projects. I've made some light comments to improve productivity without any kind of aggressive push yet have been confronted with a negative, we don't do it that way, we've done it this way, our way, for years. I'm appalled by the code quality at the same time but that's a different matter and I'm not going to reopen that extremely long thread we had in the past. If those being pushed away continue to be pushed away then the right to fork and start a new community presents an excellent opportunity for the oppressed voices. Everybody deserves a chance. AFAIC I don't care of this proposal is a complete fork of HBase itself. If they want to build a community in line with the Apache Way then we welcome them to try. If they fail to meet the requirements then that's no skin off our back. You are of course entitled to your opinions and concerns, but if you think that there is some policy that supports the position that PMC members should vote -1 on this proposal due to these issues I wish you would cite it. Ditto. SNIP ... Best Regards, -- Alex - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
I think that the ASF and ASF incubator leadership should consider it a priority to foster such communication. And this means asking the questions I asked before, e.g,. you guys are trying to do the same thing as... Have you talked them? Consider it Open-Source Project Parenting. FWIW, while I agree that it's fine to have multiple competing projects, I do very much agree with this sentiment... encouraging some (at least initial) inter-project communication would probably be a Good Thing. Who knows, it may turn out to be the case that the HBase crew are wildly interested in this stuff, and this project can be be part of HBase from day 0. Or not that, but maybe there are some natural avenues for sharing (code|design|support|whatever). At a minimum, I think encouraging the two groups to do a little chatting is a positive. Cheers, Phil - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Tue, Sep 6, 2011 at 7:41 PM, Adam P Fuchs adam.p.fu...@ugov.gov wrote: ...Furthermore, I believe that having both projects be part of ASF gives us more of an opportunity to collaborate going forwards. Whether we find that there are enough competing ideas to support two top level projects, or whether we end up collapsing Accumulo and HBase into one project, I think that the competition and collaboration we will see in the short term will be healthy for both projects. IMO, the HBase project would be more into collaborating than it would be up for competing. The differences listed in the proposal are minor IMO. If you'd come to the hbase project, even now, today, and said add a label to the KeyValue and you'll get a set of smart developers on board and a bunch of new users, it would be done. I agree w/ Doug that 'unlikely to' is not a correct characterization. Similar for Accumulo Iterators (hbase TRUNK coprocessors seem to be a more generic Iterator facility). But rumor has it though that the differences while small looking when described in a short incubator proposal, in implementation, the code is very different making an integration project, unfortunately, a piece of work. I'd be up for exploring how we can come together. I've not seen the code so excuse me my idealism. The discussion here has been enlightening. The circumstances that brought about this proposal are explained and why you could not ask the hbase project do something like add a label to KeyValue. I'm gung-ho on gov going more foss. If you lads are helping along that effort, more power to you. I'm looking forward to the code drop. That you didn't use more hbase code must mean that you came up with something better. We want to rob what you've come up with. St.Ack HBase Chief Janitor P.S. I was surprised not to see Julian Assange alongside Doug as a sponsor for this proposal - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Wednesday, September 7, 2011 1:34:20 PM, Stack st...@duboce.net wrote: I agree w/ Doug that 'unlikely to' is not a correct characterization. Would the following alteration be more accurate? It may be possible to incorporate the desired features of Accumulo into HBase. However, the amount of work required at the current time would slow development of HBase and Accumulo considerably. But rumor has it though that the differences while small looking when described in a short incubator proposal, in implementation, the code is very different making an integration project, unfortunately, a piece of work. Yes, the implementation is very different, and we had difficulty capturing that in the proposal. hbase TRUNK coprocessors seem to be a more generic Iterator facility Some types of functions (e.g. query-time aggregation) can be implemented in both coprocessors and iterators, but coprocessors will not easily support the entirety of iterator functionality. Nor is the reverse true. The two models present different programming mechanisms for server-side processing. It would be useful to have both in the same project. Also, I saw that you wondered in a different forum whether editing locality groups requires taking a table offline. The table can remain online because files contain information about their own column groupings. Thus the changes can take place as new files are written to disk and old files are compacted. Billie - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Wed, Sep 7, 2011 at 12:47 PM, Billie J Rinaldi billie.j.rina...@ugov.gov wrote: On Wednesday, September 7, 2011 1:34:20 PM, Stack st...@duboce.net wrote: I agree w/ Doug that 'unlikely to' is not a correct characterization. Would the following alteration be more accurate? It may be possible to incorporate the desired features of Accumulo into HBase. However, the amount of work required at the current time would slow development of HBase and Accumulo considerably. From my perspective, that is more the case though your second sentence above comes across as a setup for our not integrating. But rumor has it though that the differences while small looking when described in a short incubator proposal, in implementation, the code is very different making an integration project, unfortunately, a piece of work. Yes, the implementation is very different, and we had difficulty capturing that in the proposal. Understood. hbase TRUNK coprocessors seem to be a more generic Iterator facility Some types of functions (e.g. query-time aggregation) can be implemented in both coprocessors and iterators, but coprocessors will not easily support the entirety of iterator functionality. Nor is the reverse true. The two models present different programming mechanisms for server-side processing. It would be useful to have both in the same project. I'll take your word for it not having seen the code. St.Ack - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Folks, A bit of back story of a slightly similar situation before. We have had Axis/Axis2 projects in Apache for a long time and along came the XFire proposal (http://wiki.apache.org/incubator/CeltiXfireProposal) and we had the same kind of discussions as i see here. Flash forward, Both Axis2 and CXF are flourishing, there are folks who are working on components that get used by both projects like XmlSchema and WSS4J. So my 2 cents, take a deep breath and think of ways to collaborate even if it means 2 code bases with some common code, it does not have to be one code base for one situation/scenario. Do reach out to each other, hang out on each others mailing lists to make that collaboration happen. thanks, dims On Wed, Sep 7, 2011 at 4:14 PM, Stack st...@duboce.net wrote: On Wed, Sep 7, 2011 at 12:47 PM, Billie J Rinaldi billie.j.rina...@ugov.gov wrote: On Wednesday, September 7, 2011 1:34:20 PM, Stack st...@duboce.net wrote: I agree w/ Doug that 'unlikely to' is not a correct characterization. Would the following alteration be more accurate? It may be possible to incorporate the desired features of Accumulo into HBase. However, the amount of work required at the current time would slow development of HBase and Accumulo considerably. From my perspective, that is more the case though your second sentence above comes across as a setup for our not integrating. But rumor has it though that the differences while small looking when described in a short incubator proposal, in implementation, the code is very different making an integration project, unfortunately, a piece of work. Yes, the implementation is very different, and we had difficulty capturing that in the proposal. Understood. hbase TRUNK coprocessors seem to be a more generic Iterator facility Some types of functions (e.g. query-time aggregation) can be implemented in both coprocessors and iterators, but coprocessors will not easily support the entirety of iterator functionality. Nor is the reverse true. The two models present different programming mechanisms for server-side processing. It would be useful to have both in the same project. I'll take your word for it not having seen the code. St.Ack - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Davanum Srinivas :: http://davanum.wordpress.com - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Wed, Sep 7, 2011 at 5:06 PM, Davanum Srinivas dava...@gmail.com wrote: Folks, A bit of back story of a slightly similar situation before. We have had Axis/Axis2 projects in Apache for a long time and along came the XFire proposal (http://wiki.apache.org/incubator/CeltiXfireProposal) and we had the same kind of discussions as i see here. Flash forward, Both Axis2 and CXF are flourishing, there are folks who are working on components that get used by both projects like XmlSchema and WSS4J. So my 2 cents, take a deep breath and think of ways to collaborate even if it means 2 code bases with some common code, it does not have to be one code base for one situation/scenario. Do reach out to each other, hang out on each others mailing lists to make that collaboration happen. There's another useful analogy with CXF. If a group of people showed up with a proposal to make a brand new fork and launch on it, I for one would politely ask them why they feel the need to diverge, and I might hear something that disturbed me. However, that's not the situation here. A community of developers formed inside of a set of institutions that did not allow them to interact with HBase. Now they exist, and their code exists, and their rules have changed. They propose to join us. My reaction? Wonderful! It may be that, over time, this group and the existing HBase group will discover a commonality of interest and goal, and coalesce into one big happy community. It may also be that differing priorities won't lead in that direction. thanks, dims On Wed, Sep 7, 2011 at 4:14 PM, Stack st...@duboce.net wrote: On Wed, Sep 7, 2011 at 12:47 PM, Billie J Rinaldi billie.j.rina...@ugov.gov wrote: On Wednesday, September 7, 2011 1:34:20 PM, Stack st...@duboce.net wrote: I agree w/ Doug that 'unlikely to' is not a correct characterization. Would the following alteration be more accurate? It may be possible to incorporate the desired features of Accumulo into HBase. However, the amount of work required at the current time would slow development of HBase and Accumulo considerably. From my perspective, that is more the case though your second sentence above comes across as a setup for our not integrating. But rumor has it though that the differences while small looking when described in a short incubator proposal, in implementation, the code is very different making an integration project, unfortunately, a piece of work. Yes, the implementation is very different, and we had difficulty capturing that in the proposal. Understood. hbase TRUNK coprocessors seem to be a more generic Iterator facility Some types of functions (e.g. query-time aggregation) can be implemented in both coprocessors and iterators, but coprocessors will not easily support the entirety of iterator functionality. Nor is the reverse true. The two models present different programming mechanisms for server-side processing. It would be useful to have both in the same project. I'll take your word for it not having seen the code. St.Ack - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Davanum Srinivas :: http://davanum.wordpress.com - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Wednesday, September 7, 2011 4:14:26 PM, Stack st...@duboce.net wrote: From my perspective, that is more the case though your second sentence above comes across as a setup for our not integrating. How about this? It may be possible to incorporate the desired features of Accumulo into HBase. However, the amount of work required would slow development of HBase and Accumulo considerably. We believe this warrants a podling for Accumulo at the current time. We expect active cross-pollination will occur between HBase and podling Accumulo and it is possible that the codebases and projects will ultimately converge. Billie - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
On Wed, Sep 7, 2011 at 2:21 PM, Billie J Rinaldi billie.j.rina...@ugov.gov wrote: How about this? It may be possible to incorporate the desired features of Accumulo into HBase. However, the amount of work required would slow development of HBase and Accumulo considerably. We believe this warrants a podling for Accumulo at the current time. We expect active cross-pollination will occur between HBase and podling Accumulo and it is possible that the codebases and projects will ultimately converge. This is better than the former. Good luck, St.Ack - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Sounds great! -- dims On Wed, Sep 7, 2011 at 5:21 PM, Billie J Rinaldi billie.j.rina...@ugov.gov wrote: On Wednesday, September 7, 2011 4:14:26 PM, Stack st...@duboce.net wrote: From my perspective, that is more the case though your second sentence above comes across as a setup for our not integrating. How about this? It may be possible to incorporate the desired features of Accumulo into HBase. However, the amount of work required would slow development of HBase and Accumulo considerably. We believe this warrants a podling for Accumulo at the current time. We expect active cross-pollination will occur between HBase and podling Accumulo and it is possible that the codebases and projects will ultimately converge. Billie - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Davanum Srinivas :: http://davanum.wordpress.com - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Thanks Phil, much appreciated. On 9/7/11 12:19 PM, Phillip Rhodes motley.crue@gmail.com wrote: I think that the ASF and ASF incubator leadership should consider it a priority to foster such communication. And this means asking the questions I asked before, e.g,. you guys are trying to do the same thing as... Have you talked them? Consider it Open-Source Project Parenting. FWIW, while I agree that it's fine to have multiple competing projects, I do very much agree with this sentiment... encouraging some (at least initial) inter-project communication would probably be a Good Thing. Who knows, it may turn out to be the case that the HBase crew are wildly interested in this stuff, and this project can be be part of HBase from day 0. Or not that, but maybe there are some natural avenues for sharing (code|design|support|whatever). At a minimum, I think encouraging the two groups to do a little chatting is a positive. Cheers, Phil - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Doug Meil, Top-posting is generally less than idea, but I want to respond to the overall theme of your message, not to individual points and sentences. The attitude of various chunks of the US government to Open Source has changed radically over the last few years. What would have been institutionally impossible is now encouraged. It is unfair to criticize these people for being under the radar when their management told them to be under the radar and claim that the leopard cannot change its spots. That situation has changed. The code base exists, it serves a particular collection of needs, and here they are. Lots of incubator proposals come from groups of people with little or no prior track record at Apache or in Open Source at all. The point of incubation is to give them a chance to either learn how to be an Apache TLP -- or not. We do not tell people 'go away, you haven't enough pedigree.' Mentors have more work to do when the nucleus of a podling is comparatively inexperienced. Further, I don't know of any policy of the ASF in general, or the incubator in particular, that insists that a group of itch-scratchers make any particular attempt to make any particular gesture towards any particular existing community as a prerequisite of incubation. Five of my friends and I could turn up here tomorrow with a proposal to fork an existing TLP to scratch our particular itch, and, so far as I can tell, the incubator PMC would evaluate the proposal on its merits, independent of the fork. You are of course entitled to your opinions and concerns, but if you think that there is some policy that supports the position that PMC members should vote -1 on this proposal due to these issues I wish you would cite it. In the past, committers of existing TLP's have been, well, particularly vigilant in monitoring the progress of 'competing' podlings. CXF was an example podling of this effect. You are, of course, welcome to adopt that stance. You are also welcome to slurp up every line of their code, once in place, if you think that HBase could benefit from it. Perhaps needless to say, but as a Mentor-in-waiting, I'm all in favor of this proposal. --benson margulies - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
The ASF is not Highlander. If we're big enough to host a few different webservers, we're big enough for 2 HBase projects. From: Doug Meil doug.m...@explorysmedical.com To: general@incubator.apache.org general@incubator.apache.org Sent: Tuesday, September 6, 2011 8:06 PM Subject: Accumulo incubator proposal: Statement of Concern I am writing to state my concerns with the Accumulo Incubator proposal, the HBase copy/clone. 1) “Accumulo has been in development since spring 2008.” I don’t fault anybody for being scared of HBase in 2008 – you’d have to be pretty brave to use it then. HBase 0.20 was the first release to get wide adoption and that came out in the fall of 2009. That said, the fall of 2009 was two years ago. 2) “Some of the desired features of Accumulo could be incorporated into HBase, however the most important of these may be unlikely to be adopted (see cell-level access labels and iterators below)” The proposal claims that the most important features “may be unlikely to be adopted” by HBase. Really?? How do the Accumulo developers know this? Not a single request was made either in dist-list or Jira form to the HBase community regarding these requested features. Why is open communication such a problem? Remember that Accumulo had 2 years to put together such a request. For a project trying to achieve the exact same goals as HBase, this is not a minor issue. The past is unfortunately the best predictor of futureperformance, and while excuses have been made about sharing code and communication being “hard” for the employer of the majority of the Accumulo developers, the lack of open-ness for an ASF project is a non-starter. For example, the HBase team received a recent finger-wag when the project committers voted on a new logo (i.e., instead of letting the entire community vote). This somewhat humorous infraction does prove the point: open-ness is required at all levels. HBase has, for years, demonstrated this principle through public feature requests, public bug reports, public code reviews, and public dist-list conversations on every conceivable issue. Based on past performance, I don’t see Accumulo, or the developers behind Accumulo, being able to make the cut in this respect. 3) “ === Apache Brand === Our interest in releasing this code as an Apache incubator project is due to its strong relationship with other Apache projects, i.e. Hadoop, Zookeeper, and HBase” Regarding “strong relationship” see point #2 about on non-communication over the last few years. Side-bar conversations with a Hadoop developer or two do not count as “community communication.” 4) In Summary If Accumulo wishes to be an open-source project, so be it - but put it on Google Code, SourceForge, or Github. There are plenty of places. But I don’t think it belongs in ASF. I’m sure that other developers may have some comments about copied HBase and Hadoop code, but I’ll leave that to them. Sincerely, Doug Meil
Re: Accumulo incubator proposal: Statement of Concern
Re: Highlander Greatest movie ever. re: Webservers This is a good point, but a closer inspection of the webservers shows that the situation is quite different from Accumulo vs. HBase. Apache WS is in implemented in C, and Tomcat is implemented in Java. Accumulo is, effectively, trying to be Timdog - by being the 3rd webserver that is just like Tomcat, and in fact borrows some of the code from Tomcat (another issue, but point of fact true) and also claims to have a community relationship with Tomcat (which nobody in the HBase community has seen). On 9/6/11 8:46 PM, Joe Schaefer joe_schae...@yahoo.com wrote: The ASF is not Highlander. If we're big enough to host a few different webservers, we're big enough for 2 HBase projects. From: Doug Meil doug.m...@explorysmedical.com To: general@incubator.apache.org general@incubator.apache.org Sent: Tuesday, September 6, 2011 8:06 PM Subject: Accumulo incubator proposal: Statement of Concern I am writing to state my concerns with the Accumulo Incubator proposal, the HBase copy/clone. 1) ³Accumulo has been in development since spring 2008.² I don¹t fault anybody for being scared of HBase in 2008 you¹d have to be pretty brave to use it then. HBase 0.20 was the first release to get wide adoption and that came out in the fall of 2009. That said, the fall of 2009 was two years ago. 2) ³Some of the desired features of Accumulo could be incorporated into HBase, however the most important of these may be unlikely to be adopted (see cell-level access labels and iterators below)² The proposal claims that the most important features ³may be unlikely to be adopted² by HBase. Really?? How do the Accumulo developers know this? Not a single request was made either in dist-list or Jira form to the HBase community regarding these requested features. Why is open communication such a problem? Remember that Accumulo had 2 years to put together such a request. For a project trying to achieve the exact same goals as HBase, this is not a minor issue. The past is unfortunately the best predictor of futureperformance, and while excuses have been made about sharing code and communication being ³hard² for the employer of the majority of the Accumulo developers, the lack of open-ness for an ASF project is a non-starter. For example, the HBase team received a recent finger-wag when the project committers voted on a new logo (i.e., instead of letting the entire community vote). This somewhat humorous infraction does prove the point: open-ness is required at all levels. HBase has, for years, demonstrated this principle through public feature requests, public bug reports, public code reviews, and public dist-list conversations on every conceivable issue. Based on past performance, I don¹t see Accumulo, or the developers behind Accumulo, being able to make the cut in this respect. 3) ³ === Apache Brand === Our interest in releasing this code as an Apache incubator project is due to its strong relationship with other Apache projects, i.e. Hadoop, Zookeeper, and HBase² Regarding ³strong relationship² see point #2 about on non-communication over the last few years. Side-bar conversations with a Hadoop developer or two do not count as ³community communication.² 4) In Summary If Accumulo wishes to be an open-source project, so be it - but put it on Google Code, SourceForge, or Github. There are plenty of places. But I don¹t think it belongs in ASF. I¹m sure that other developers may have some comments about copied HBase and Hadoop code, but I¹ll leave that to them. Sincerely, Doug Meil - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Greetings Benson, re: top-posting is generally less than ideal It's funny that you would say that, because there was a 17 message email-chain about Accumulo this past weekend that you participated in, and I noticed that you didn't tell any of those people that posting on this dist-list was a bad idea. http://mail-archives.apache.org/mod_mbox/incubator-general/201109.mbox/brow ser re: pedigree Pedigree is hardly what anybody is looking for. I have been answering questions on the HBase dist-list for 2 years (yes, since HBase 0.20) and I can tell you personally that some of the people on the dist-list are brilliant, and others I wonder how they managed to turn their computer on (hey, did I just say that out loud?). But we try to answer all their questions just the same. I'll let the other HBase team members speak for themselves, but my request to ASF is that if somebody proposes something that nearly exactly like an existing ASF project, and it's implemented in the same language, and it even copies their code - shouldn't somebody at ASF ask the proposing team: have you guys talked to the other project? Especially since, in this case, one of their claims is that HBase can't do or has no plans for something. It's all about community, isn't it? On 9/6/11 8:27 PM, Benson Margulies bimargul...@gmail.com wrote: Doug Meil, Top-posting is generally less than idea, but I want to respond to the overall theme of your message, not to individual points and sentences. The attitude of various chunks of the US government to Open Source has changed radically over the last few years. What would have been institutionally impossible is now encouraged. It is unfair to criticize these people for being under the radar when their management told them to be under the radar and claim that the leopard cannot change its spots. That situation has changed. The code base exists, it serves a particular collection of needs, and here they are. Lots of incubator proposals come from groups of people with little or no prior track record at Apache or in Open Source at all. The point of incubation is to give them a chance to either learn how to be an Apache TLP -- or not. We do not tell people 'go away, you haven't enough pedigree.' Mentors have more work to do when the nucleus of a podling is comparatively inexperienced. Further, I don't know of any policy of the ASF in general, or the incubator in particular, that insists that a group of itch-scratchers make any particular attempt to make any particular gesture towards any particular existing community as a prerequisite of incubation. Five of my friends and I could turn up here tomorrow with a proposal to fork an existing TLP to scratch our particular itch, and, so far as I can tell, the incubator PMC would evaluate the proposal on its merits, independent of the fork. You are of course entitled to your opinions and concerns, but if you think that there is some policy that supports the position that PMC members should vote -1 on this proposal due to these issues I wish you would cite it. In the past, committers of existing TLP's have been, well, particularly vigilant in monitoring the progress of 'competing' podlings. CXF was an example podling of this effect. You are, of course, welcome to adopt that stance. You are also welcome to slurp up every line of their code, once in place, if you think that HBase could benefit from it. Perhaps needless to say, but as a Mentor-in-waiting, I'm all in favor of this proposal. --benson margulies - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Accumulo incubator proposal: Statement of Concern
Hello Doug, I appreciate your concerns, and I would like to try to provide some context that might help you understand why we think the ASF is the right place for Accumulo. Benson is correct that the Accumulo developers have historically not been authorized to participate in public open-source forums in any official capacity. This has led us to developing Accumulo as an internal open-source project rather than participating directly in the development of HBase or other BigTable derivatives. While we have been coding Accumulo for the past three years, we have also been working to improve our policies for a good portion of that time. We expect that our policy efforts, which have enabled us to make this proposal to ASF, will enable us to participate in the open source community in a way that was not previously possible. We're also hoping that our participation will draw more open-source participation from a large group of developers working in other government organizations and contractors. While our internal group is only a microcosm of the true, global open-source community, we have still been operating with good open-source coding practices. You don't really have to take our word for that -- the incubator process gives us a great opportunity to show that we can properly participate. I understand that you're also concerned that Accumulo will compete for resources with HBase, and that there might not be room for the two projects in ASF. I would argue that development resources will be (and have been) split between the two projects regardless of whether they are both part of ASF or not. Furthermore, I believe that having both projects be part of ASF gives us more of an opportunity to collaborate going forwards. Whether we find that there are enough competing ideas to support two top level projects, or whether we end up collapsing Accumulo and HBase into one project, I think that the competition and collaboration we will see in the short term will be healthy for both projects. I look forward to working with you more closely in the future. Cheers, Adam P.S. We didn't mean to imply in the proposal that we have a relationship with the community developing HBase -- only to imply that Accumulo and HBase share common goals. We have reworded the proposal to try to clarify this. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
