Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Hi, On Wed, Mar 28, 2012 at 3:35 PM, Roy T. Fielding field...@gbiv.com wrote: Yes, it's called a -deps package, and individuals occasionally produce them and even redistribute them from our servers (as binaries). So, to move this discussion forward, do you think it would be acceptable if ManifoldCF (and any other project with binary dependencies) simply moved the binaries away from the source package to an associated -deps package that can be installed on top of the source tree to make it buildable? BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Fri, Mar 30, 2012 at 10:34 AM, Jukka Zitting jukka.zitt...@gmail.comwrote: Hi, On Wed, Mar 28, 2012 at 3:35 PM, Roy T. Fielding field...@gbiv.com wrote: Yes, it's called a -deps package, and individuals occasionally produce them and even redistribute them from our servers (as binaries). So, to move this discussion forward, do you think it would be acceptable if ManifoldCF (and any other project with binary dependencies) simply moved the binaries away from the source package to an associated -deps package that can be installed on top of the source tree to make it buildable? I think that Roy already gave a clear answer of 'yes' to this in reply to my previous message BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Hi, On Fri, Mar 30, 2012 at 4:37 PM, Benson Margulies bimargul...@gmail.com wrote: On Fri, Mar 30, 2012 at 10:34 AM, Jukka Zitting jukka.zitt...@gmail.comwrote: So, to move this discussion forward, do you think it would be acceptable if ManifoldCF (and any other project with binary dependencies) simply moved the binaries away from the source package to an associated -deps package that can be installed on top of the source tree to make it buildable? I think that Roy already gave a clear answer of 'yes' to this in reply to my previous message Let's go with that. I'll ask the podling to recut a release candidate with this (or some equivalent) change in place. BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Hi, On Wed, Mar 28, 2012 at 5:19 PM, Leo Simons m...@leosimons.com wrote: Shipping a set of CDDL jars out of some java.net projects that oracle has all but abandoned is far beyond my imagined threshold of reasonable on the scale. Do you actually see that differently? Agreed. These are exactly the kinds of questions that legal-discuss@ has been working on. I.e. which kinds of dependencies are acceptable, and how they should be referenced/included/documented/etc.? It seems like Roy is much more categorical about this. Assuming I understand his point correctly, *no* binary dependencies are acceptable within a source tarball. What I don't quite (yet) understand is how a reference like junit:junit:4.10 to a download service maintained by a third party is more acceptable than directly including the referenced bits. The only difference I see is whether we have the right to redistribute those bits ourselves, but that question is already covered by legal. Another thing I don't understand is how a configure script compiled by autoconf from sources in configure.in and the autoconf distribution is any less a binary artifact than a dependency jar. Reviewing a ~1MB configure script is about as easy as reviewing the decompiled contents of a binary jar. That's a lot I don't understand. If this is as clear an issue as is being claimed, I'm sure someone will jump in to educate me. What is the alternative you're thinking of? Is it merely about the process by which we clean things up, or is there some other kind of more fundamental alternative? The obvious alternative would be to bless the de facto practice from the past 10+ years as being acceptable also de jure. BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Thu, Mar 29, 2012 at 2:41 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: ...It seems like Roy is much more categorical about this. Assuming I understand his point correctly, *no* binary dependencies are acceptable within a source tarball. What I don't quite (yet) understand is how a reference like junit:junit:4.10 to a download service maintained by a third party is more acceptable than directly including the referenced bits... I think the difference is that by saying get junit:junit:4.10 to build this we put the burden on our users to make sure they get the right bits, either by building them themselves from the junit sources, or trusting whoever provides them. By shipping those bits ourselves instead, we would take the responsibility on our shoulders, which we don't want. -Bertrand - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
I mentioned this in another note but I'll repeat here to use your example. Where the binaries do live in a Maven repo and are versioned there is less of an issue and it becomes a convenience. A real challenge is what to do if your taking a stable copy of a project that doesn't have a versioned release? The only method I know of is to capture their source, and build a binary at a stable level to include in your projects release. Matt Hogstrom m...@hogstrom.org A Day Without Nuclear Fusion Is a Day Without Sunshine On Mar 29, 2012, at 9:07 AM, Bertrand Delacretaz wrote: On Thu, Mar 29, 2012 at 2:41 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: ...It seems like Roy is much more categorical about this. Assuming I understand his point correctly, *no* binary dependencies are acceptable within a source tarball. What I don't quite (yet) understand is how a reference like junit:junit:4.10 to a download service maintained by a third party is more acceptable than directly including the referenced bits... I think the difference is that by saying get junit:junit:4.10 to build this we put the burden on our users to make sure they get the right bits, either by building them themselves from the junit sources, or trusting whoever provides them. By shipping those bits ourselves instead, we would take the responsibility on our shoulders, which we don't want. -Bertrand - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Jukka Zitting wrote on Thu, Mar 29, 2012 at 14:41:02 +0200: Hi, On Wed, Mar 28, 2012 at 5:19 PM, Leo Simons m...@leosimons.com wrote: Shipping a set of CDDL jars out of some java.net projects that oracle has all but abandoned is far beyond my imagined threshold of reasonable on the scale. Do you actually see that differently? Agreed. These are exactly the kinds of questions that legal-discuss@ has been working on. I.e. which kinds of dependencies are acceptable, and how they should be referenced/included/documented/etc.? It seems like Roy is much more categorical about this. Assuming I understand his point correctly, *no* binary dependencies are acceptable within a source tarball. What I don't quite (yet) understand is how a reference like junit:junit:4.10 to a download service maintained by a third party is more acceptable than directly including the referenced bits. The only difference I see is whether we have the right to redistribute those bits ourselves, but that question is already covered by legal. junit is only needed for unit tests and not for the software itself; is this relevant to the example? Another thing I don't understand is how a configure script compiled by autoconf from sources in configure.in and the autoconf distribution is any less a binary artifact than a dependency jar. Reviewing a ~1MB configure script is about as easy as reviewing the decompiled contents of a binary jar. Have you tried it? At Subversion we use the same autoconf throughout a release branch, so comparing two successive 1.6.x tarballs (expanded tarballs, which include compiled swig and and configure code) wasn't impossible. And for 1.6.0, had I wanted to, I could have run the rolling script myself to generate my own versions of the tarballs locally, and compare those. (Hmm, did you mean 'configure' just as an example? :-)) That's a lot I don't understand. If this is as clear an issue as is being claimed, I'm sure someone will jump in to educate me. What is the alternative you're thinking of? Is it merely about the process by which we clean things up, or is there some other kind of more fundamental alternative? The obvious alternative would be to bless the de facto practice from the past 10+ years as being acceptable also de jure. BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Mar 29, 2012, at 15:07 , Bertrand Delacretaz wrote: On Thu, Mar 29, 2012 at 2:41 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: ...It seems like Roy is much more categorical about this. Assuming I understand his point correctly, *no* binary dependencies are acceptable within a source tar ball. Let's see if I still correctly follow this discussion. So far we seem to have consensus about the fact that: a) the only official releases that Apache does are source releases, and b) source releases must not contain binaries (of any dependencies). So far so good, and the only suggestion I have in this area is that we should make a more clear distinction between what we officially release (and vote on) and anything else we might provide for convenience. Just taking a look at www.apache.org/dist/ reveals that it contains both, and a lot of the time not clearly separated, which is confusing. Furthermore, it seems that some projects have more than just their latest release there (which is another matter, not related to this discussion). I propose something like: * www.apache.org/dist/[project]/ for the latest source release that was voted on * www.apache.org/bin/[project]/ for convenience binaries, etc. What I don't quite (yet) understand is how a reference like junit:junit:4.10 to a download service maintained by a third party is more acceptable than directly including the referenced bits... I think the difference is that by saying get junit:junit:4.10 to build this we put the burden on our users to make sure they get the right bits, either by building them themselves from the junit sources, or trusting whoever provides them. By shipping those bits ourselves instead, we would take the responsibility on our shoulders, which we don't want. Since we are allowed to somehow reference an artifact (as long as it has a license that is compatible with what we do) and have a build script download it, my question is, must this artifact come from a location *outside* of Apache, or are we also allowed to reference these binaries that were provided for convenience by our own projects? Related, how about binaries that are in a separate part of the SVN tree of a project (a part that is not released)? Can we reference and download (or checkout) those as part of a build script? Greetings, Marcel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Le 3/29/12 3:41 PM, Daniel Shahaf a écrit : Jukka Zitting wrote on Thu, Mar 29, 2012 at 14:41:02 +0200: Hi, On Wed, Mar 28, 2012 at 5:19 PM, Leo Simonsm...@leosimons.com wrote: Shipping a set of CDDL jars out of some java.net projects that oracle has all but abandoned is far beyond my imagined threshold of reasonable on the scale. Do you actually see that differently? Agreed. These are exactly the kinds of questions that legal-discuss@ has been working on. I.e. which kinds of dependencies are acceptable, and how they should be referenced/included/documented/etc.? It seems like Roy is much more categorical about this. Assuming I understand his point correctly, *no* binary dependencies are acceptable within a source tarball. What I don't quite (yet) understand is how a reference like junit:junit:4.10 to a download service maintained by a third party is more acceptable than directly including the referenced bits. The only difference I see is whether we have the right to redistribute those bits ourselves, but that question is already covered by legal. junit is only needed for unit tests and not for the software itself; is this relevant to the example? We have *many* external depencies in Directory (like antlr, xpp3, dom4j, openSymphony, councycastle, ...) which are stored and managed by Maven. When you build the project, all those jars will be pulled from the repository, and injected into the binary resulting from the build. If someone pull the sources from SVN, and build the project, he will get the complete working binary. One more step, and he will also get the installers (we have one sub-project that build those installers for each platform we are supporting - currently windows, linux, mac OSX, and a standard jar -. So far, so good. Now, building the project is just a nightmare for our users, so we provide the binary installers. So are most of the Java TLP, AFAICT, and thos binary contains all those external jars. My understanding is that, as far as we offer our users a way to build the binaries from svn, and as far as we don't have included the jars in SVN, we are golden. And My understanding is that this is *the release*. OTOH, the binaries we provide, ie the installers/jars/whatever are just convenient deliveries that are not blessed by The ASF. Those users who chose to pick those binaries, and expect The ASF to protect themselves against a trial because the project has badly included a binary that is not compatible with the ASL 2.0 licence, will not be protected by the ASL 2.0 licence. Now, the maven repository being hosted at The ASF, the difference is, imho, really really thin. Am I missing something ? -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On 29 March 2012 15:09, Marcel Offermans marcel.offerm...@luminis.nl wrote: On Mar 29, 2012, at 15:07 , Bertrand Delacretaz wrote: On Thu, Mar 29, 2012 at 2:41 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: ...It seems like Roy is much more categorical about this. Assuming I understand his point correctly, *no* binary dependencies are acceptable within a source tar ball. Let's see if I still correctly follow this discussion. So far we seem to have consensus about the fact that: a) the only official releases that Apache does are source releases, and b) source releases must not contain binaries (of any dependencies). So far so good, and the only suggestion I have in this area is that we should make a more clear distinction between what we officially release (and vote on) and anything else we might provide for convenience. Just taking a look at www.apache.org/dist/ reveals that it contains both, and a lot of the time not clearly separated, which is confusing. Furthermore, it seems that some projects have more than just their latest release there (which is another matter, not related to this discussion). I propose something like: * www.apache.org/dist/[project]/ for the latest source release that was voted on * www.apache.org/bin/[project]/ for convenience binaries, etc. An alternative, which many projects already use, would be: * www.apache.org/dist/[project]/source/ for the latest source release that was voted on * www.apache.org/dist/[project]/binaries/ for convenience binaries, etc. This makes it a bit easier to find the binaries from the source and vice-versa. What I don't quite (yet) understand is how a reference like junit:junit:4.10 to a download service maintained by a third party is more acceptable than directly including the referenced bits... I think the difference is that by saying get junit:junit:4.10 to build this we put the burden on our users to make sure they get the right bits, either by building them themselves from the junit sources, or trusting whoever provides them. By shipping those bits ourselves instead, we would take the responsibility on our shoulders, which we don't want. Since we are allowed to somehow reference an artifact (as long as it has a license that is compatible with what we do) and have a build script download it, my question is, must this artifact come from a location *outside* of Apache, or are we also allowed to reference these binaries that were provided for convenience by our own projects? Related, how about binaries that are in a separate part of the SVN tree of a project (a part that is not released)? Can we reference and download (or checkout) those as part of a build script? Greetings, Marcel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Hi, On Wed, Mar 28, 2012 at 1:16 AM, Leo Simons m...@leosimons.com wrote: That said, I'm not aware of us actually having such a release out there? Take such fringe projects like Ant, Tomcat, Lucene and Xalan that have been shipping releases like throughout the past decade. See examples dating back at least to [1], [2], [3] and [4]. It looks like at least Ant and Tomcat have since opted for downloading dependencies at build time, but I must have missed the memo if that change was driven by Apache policy requirements rather than just the emergence of the central Maven repository making such downloads practical. So to have anyone call this *not* a standard practice at ASF makes, to borrow your expression, my jaw drop. And a board member who's threatening to wipe out a decade worth of releases of some of the most popular open source projects in the world. Again, to borrow your expression, huh? As said earlier, I'm fine if people want to clarify our policies regarding this and revise current practice. But let's please have a proper debate about the merits of the alternatives and give affected projects proper time to adjust in case changes are required. [1] http://archive.apache.org/dist/ant/source/jakarta-ant-1.1.zip [2] http://archive.apache.org/dist/tomcat/tomcat-3/src/jakarta-tomcat-3.3.1a-src.tar.gz [3] http://archive.apache.org/dist/lucene/java/archive/lucene-1.4.3-src.tar.gz [4] http://archive.apache.org/dist/xml/xalan-j/source/xalan-j_2_2-src.tar.gz BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Roy, Of course you, personally, can't be expected to supervise all projects or fix all documentation. At the same time, there's something a little depressing about the situation. On the one hand, the principle at work here is, to paraphrase you, absolutely central to the defined mission of the Foundation. On the other hand, over all these years, the Foundation has apparently failed to do two things: to find a way to document this so clearly that no member, let alone PMC chair let alone committer could miss it; and to supervise projects sufficiently to detect divergence. The board can read reports all year long and never discover that someone has drifted off the rails here. None of this, however, is what I really want to write about. Consider me in the role of a PMC member voting on a release with external dependencies, not included (of course) in the bundle. What do I do? Well, I fetch the dependency. In source? In binary? From where? And how do I ensure that I get exactly the same thing that the next voter over gets? If the build automatically downloads, I don't appear to have this problem, but, really, what do I know about what that download is downloading? This all boils down to the semantics of the vote. All I can really do is attest that the sources are legitimate Apache sources, and that I was able to build and run something using *some version of some dependencies*. Is this really good enough? In our role as serving the public good, is this giving the user community enough information? Consider the following thought experiment: what if projects bundled up their binary dependencies into an archive with a manifest file that described their provenance, signed them, and put them someplace? Someplace not 'inside a release'. Then, the source release would be aligned with a particular, reproducible, version of its dependencies. If this is still completely unacceptable, the logic seems to lead inexorable to dealing in Other People's Sources: capturing a snapshot of their sources so as to be able to state, unequivocally, what the dependencies are. Maybe all this reads as pointless quibbling and no one cares about it. I have another issues to raise, however: the gap between public perception of Apache releases and legal reality. When AOO releases, soon, a giant flood of *binary packages* will move out onto the mirrors. Accompanied, yes, by a source release. Formally, legally, the only real release is the source package. Realistically, no end users will touch it. From their point of view, the thing with the Apache seal of approval that they will trust, download, and install will be a binary. AOO isn't unique, but it's the starkest example, because of it's end-user focus. I feel a little bit disingenuous, in our role as a public charity, about the disparity between the public perception that we're releasing binary packages and the facts. This strikes me, in unhelpful retrospect, as an issue that the board or membership should have taken up as part of accepting AOO. I don't have a proposed solution; please don't mistake me for proposing to tamper with the fundamental 'releases are source' principle. But I think that something here does not fit. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Mar 28, 2012, at 2:39 PM, Benson Margulies wrote: Roy, Of course you, personally, can't be expected to supervise all projects or fix all documentation. At the same time, there's something a little depressing about the situation. On the one hand, the principle at work here is, to paraphrase you, absolutely central to the defined mission of the Foundation. On the other hand, over all these years, the Foundation has apparently failed to do two things: to find a way to document this so clearly that no member, let alone PMC chair let alone committer could miss it; and to supervise projects sufficiently to detect divergence. The board can read reports all year long and never discover that someone has drifted off the rails here. None of this, however, is what I really want to write about. Consider me in the role of a PMC member voting on a release with external dependencies, not included (of course) in the bundle. What do I do? If you want to do it right, build the whole thing from scratch -- nothing but the source code. If there isn't at least one person (or CI bot) doing that per project, we're screwed. Well, I fetch the dependency. In source? In binary? From where? And how do I ensure that I get exactly the same thing that the next voter over gets? If the build automatically downloads, I don't appear to have this problem, but, really, what do I know about what that download is downloading? You don't. Fortunately, we require three +1s, so it is fairly hard to trick the whole PMC. This all boils down to the semantics of the vote. All I can really do is attest that the sources are legitimate Apache sources, and that I was able to build and run something using *some version of some dependencies*. Is this really good enough? Yes. In our role as serving the public good, is this giving the user community enough information? The user community is not our immediate concern. Downstream developers are. We are only responsible for what we decide to release. Consider the following thought experiment: what if projects bundled up their binary dependencies into an archive with a manifest file that described their provenance, signed them, and put them someplace? Someplace not 'inside a release'. Then, the source release would be aligned with a particular, reproducible, version of its dependencies. Yes, it's called a -deps package, and individuals occasionally produce them and even redistribute them from our servers (as binaries). Users are warned that binaries are provided for their convenience, and that building from source is preferred for verification. Organizations or individuals that would be offended by (or prevented from receiving) the binaries are fully capable of building their own IF and ONLY IF the binaries do not exist in our source packages. If this is still completely unacceptable, the logic seems to lead inexorable to dealing in Other People's Sources: capturing a snapshot of their sources so as to be able to state, unequivocally, what the dependencies are. Maybe all this reads as pointless quibbling and no one cares about it. I have another issues to raise, however: the gap between public perception of Apache releases and legal reality. When AOO releases, soon, a giant flood of *binary packages* will move out onto the mirrors. Accompanied, yes, by a source release. Formally, legally, the only real release is the source package. Realistically, no end users will touch it. From their point of view, the thing with the Apache seal of approval that they will trust, download, and install will be a binary. AOO isn't unique, but it's the starkest example, because of it's end-user focus. I feel a little bit disingenuous, in our role as a public charity, about the disparity between the public perception that we're releasing binary packages and the facts. This strikes me, in unhelpful retrospect, as an issue that the board or membership should have taken up as part of accepting AOO. I don't have a proposed solution; please don't mistake me for proposing to tamper with the fundamental 'releases are source' principle. But I think that something here does not fit. How is that different from any of our other projects? End users don't compile Java. Hell, most developers don't compile Java. We distribute plenty of binaries. We just don't call them SOURCE. The source is what we review. The source is what we bless. If anyone wants to go further than that, they are free to do so as long as they don't call the result an Apache release. It is a binary package, a user convenience, a download hosted by openoffice.org. I don't care. People have to understand this. There will always be a role for downstream commercial or non-commercial redistributions of Apache products. Why? Because the ASF is incapable of taking on the enormous liability associated with released binaries that are not produced in a controlled environment with a controlled set
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Hi, On Wed, Mar 28, 2012 at 1:59 PM, Roy T. Fielding field...@gbiv.com wrote: Me not happy. This is not any individual's fault, least of all Jukka, and certainly not a fault of the ManifoldCF podling that is going through this fun precisely to learn how to create an Apache release. It is an institutional fault. We need to separate our policy docs into example procedures and basic principles. The former are ways to do things right that can be improved upon over time. The latter are requirements established by the membership of the ASF, enshrined in our corporate mission, and enforced by the board of directors. Agreed. Since this is a significant issue with a major impact on many TLPs, I think it would be useful to have a board resolution that clarifies the issue and what should be done about it. Anyone willing to draft one? I'd also like to see a wider discussion about this, involving at least legal, infra, and the affected TLPs. What should be done about past releases? How does this affect documentation under www.apache.org/legal? etc. Anyone willing to drive this? BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Tue, Mar 27, 2012 at 1:16 PM, Roy T. Fielding field...@gbiv.com wrote: On Mar 27, 2012, at 12:57 PM, Jukka Zitting wrote: Hi, [dropped infra@, I believe most interested people are already on general@] Let's decouple this thread from the specific issue of the ManifoldCF release. There's a long tradition of Apache releases like the ones ManifoldCF is producing, so turning this suddenly into a blocker is IMHO bad business, especially since no legal issues are involved (this is about Apache policy). If we do come to the consensus that releases like this are contrary to Apache policy, then affected projects should be given a reasonable amount of time to adapt. I don't see where you get the idea that there is a long tradition of including binary artifacts within the source package releases at Apache. There may be specific groups who are apparently lacking the appropriate clue and stubbornly refuse to read the messages I have sent multiple times to this mailing list, legal-discuss, and members, but there is no question whatsoever that a source package cannot include binaries. It would not be a source package otherwise. I think this may be overstating things. The issue should be lack of source code, not presence of binary code. For example, I could have a Java code that relies on a native method implemented in C code. I could have a source package that contains the complete Java and the complete C code, all under ALv2. But do we really want to say that we cannot also include, in the source page, the native code, pre-compiled as a convenience for the developer? The alternative would be that a downstream developer who is modifying only unrelated Java portions of the source code would be required to compile the native code on all platforms in order to create a package. (It would also require the PMC to have rather elaborate build rituals to create that JAR, since it would require that we shuffle libraries across multiple buildbots) -Rob
RE: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
I think a bigger issue specifically for Apache OpenOffice has to do with branding and status of the released binaries. It is where the Apache (plus incubating) origin is brought to the users attention and what will be understood for it. It is served up directly (from the user perspective) from www.openoffice.org and the rebranding of that site as under Apache (plus incubating) custodianship. There is in no sense an indication that this is a downstream product of the formal Apache source-code release. I suspect that ordinary users would be baffled by such a distinction and find it difficult to see why that is significant. To the extent I understand it, I can sympathize with what Roy Fielding says about provenance and reproducibility. I just don't know how the catch-22 here can be dealt with. - Dennis AN ILL-CHOSEN THOUGHT-PROBLEM? Perhaps not a good example, but one that still nags me in wondering how I would handle it on a non-ASF project, especially in the context of Fielding's maxim, is that it has finally been considered all right to bundle GPL-origin artifacts (spelling dictionary, its index, and any thesaurus) in AOO binary distributions. There appears to me -- and I am not expert in this matter -- that there is no evident means to build those artifacts from what the GPL (and the OSD) deems the source code to be. The original distributors have not seen fit to indicate where that can be found (if it was ever made available). Some claim these artifacts are their own sources, being essentially coded data, and the provision of a reliable location not in the Apache SVN for obtaining them may suffice here. If so, I suspect that modifications carried out to correct and modernize these artifacts needs to be under a same-license project somewhere so that the license conditions on the derivatives are satisfied, including combining into OpenOffice extension artifacts and making them readily available. (They are not readily extracted from the AOO bundle and the resulting installed software, although it is technically possible to separate them out if one knows where to look.) Such a project, if that is the appropriate resolution, needs to be meticulous with regard to the license and having its license-honoring artifacts reliably available. It would appear that must be done elsewhere (but not on Apache Extras?). I suppose it is valuable, in this context, that SourceForge has been helpful with related matters. -Original Message- From: Rob Weir [mailto:robw...@apache.org] Sent: Wednesday, March 28, 2012 08:26 To: general@incubator.apache.org Subject: Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0) On Tue, Mar 27, 2012 at 1:16 PM, Roy T. Fielding field...@gbiv.com wrote: On Mar 27, 2012, at 12:57 PM, Jukka Zitting wrote: Hi, [dropped infra@, I believe most interested people are already on general@] Let's decouple this thread from the specific issue of the ManifoldCF release. There's a long tradition of Apache releases like the ones ManifoldCF is producing, so turning this suddenly into a blocker is IMHO bad business, especially since no legal issues are involved (this is about Apache policy). If we do come to the consensus that releases like this are contrary to Apache policy, then affected projects should be given a reasonable amount of time to adapt. I don't see where you get the idea that there is a long tradition of including binary artifacts within the source package releases at Apache. There may be specific groups who are apparently lacking the appropriate clue and stubbornly refuse to read the messages I have sent multiple times to this mailing list, legal-discuss, and members, but there is no question whatsoever that a source package cannot include binaries. It would not be a source package otherwise. I think this may be overstating things. The issue should be lack of source code, not presence of binary code. For example, I could have a Java code that relies on a native method implemented in C code. I could have a source package that contains the complete Java and the complete C code, all under ALv2. But do we really want to say that we cannot also include, in the source page, the native code, pre-compiled as a convenience for the developer? The alternative would be that a downstream developer who is modifying only unrelated Java portions of the source code would be required to compile the native code on all platforms in order to create a package. (It would also require the PMC to have rather elaborate build rituals to create that JAR, since it would require that we shuffle libraries across multiple buildbots) -Rob - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Mar 28, 2012, at 9:35 AM, Roy T. Fielding wrote: If you want to do it right, build the whole thing from scratch -- nothing but the source code. If there isn't at least one person (or CI bot) doing that per project, we're screwed. I think the problem has gotten more challenging over time as many projects (at least the Java related ones) have a large number of dependencies on other Java projects. The examples like Ant are good. I'll point out Geronimo and a number of the other open source projects that build around JEE. There is no JEE project per se, there are lots of different implementations that get woven together. Geronimo is probably hardest hit because the project had to include dependencies from many other projects. In some cases, the project took snapshots from the other projects in order to ship because not all projects release in sync. To avoid the problem, at least a few years ago, we built a repo where we would capture the maven artifacts so a Geronimo release could be built with a set of known and versioned dependencies. To provide any sense of repeatability this practice was necessary. Perhaps we need a clarification on wording. We have a release and we have distributions. The release is the vote on the source of the project and the distributions are a versioned source tar-ball plus other binaries for different platforms or configurations. We do release source and we do distribute binaries and source. In some cases, the source contains binaries which are dependencies but in no case that I'm aware of are the binaries not from an open, referenceable and verifiable open source project. Matt Hogstrom m...@hogstrom.org A Day Without Nuclear Fusion Is a Day Without Sunshine - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
Hi, [dropped infra@, I believe most interested people are already on general@] Let's decouple this thread from the specific issue of the ManifoldCF release. There's a long tradition of Apache releases like the ones ManifoldCF is producing, so turning this suddenly into a blocker is IMHO bad business, especially since no legal issues are involved (this is about Apache policy). If we do come to the consensus that releases like this are contrary to Apache policy, then affected projects should be given a reasonable amount of time to adapt. Also, let's make a clear distinction between binary distributions (i.e. the packages that result from building one of our source releases) and binary dependencies (i.e. binary distributions of upstream projects). AFAICT there's clear consensus that binary distributions are *not* official Apache releases, and we've been pretty consistent about that so far. However, the word on binary dependencies is much less clear. There's explicit Apache policy (category B, etc.) that talks about binary dependencies and plenty of Apache releases contain them. This is clearly not an area where we have consensus. On Tue, Mar 27, 2012 at 11:50 AM, Roy T. Fielding field...@gbiv.com wrote: Likewise for jar files of dependencies -- they are NOT our product and they MUST NOT be present in the source code package that is voted on for release. Citation needed. Note that the source materials reference you mentioned earlier is vague. It covers stuff like configure scripts in httpd releases, test files, and indeed (as far as it so far has been interpreted) binary dependencies of upstream open source projects. I'm fine if this point needs to be clarified and some current practice terminated, but let's follow standard process to do so. If any ASF member is aware of an Apache release package that is not 100% open source code, you are hereby instructed to DELETE it from our servers. What hat are you holding here? Which packages explicitly are you referring to? BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Mar 27, 2012, at 12:57 PM, Jukka Zitting wrote: Hi, [dropped infra@, I believe most interested people are already on general@] Let's decouple this thread from the specific issue of the ManifoldCF release. There's a long tradition of Apache releases like the ones ManifoldCF is producing, so turning this suddenly into a blocker is IMHO bad business, especially since no legal issues are involved (this is about Apache policy). If we do come to the consensus that releases like this are contrary to Apache policy, then affected projects should be given a reasonable amount of time to adapt. I don't see where you get the idea that there is a long tradition of including binary artifacts within the source package releases at Apache. There may be specific groups who are apparently lacking the appropriate clue and stubbornly refuse to read the messages I have sent multiple times to this mailing list, legal-discuss, and members, but there is no question whatsoever that a source package cannot include binaries. It would not be a source package otherwise. http://incubator.apache.org/guides/releasemanagement.html This issue is not open for discussion. It is is a mandate from the certificate of this foundation -- our agreement with the State of Delaware that I signed as incorporator. It is fundamental to our status as an IRS 501(c)3 charity. It is the key charter delegated by the board as part of every TLP resolution: charged with the creation and maintenance of open-source software ... for distribution at no charge to the public. Class files are not open source. Jar files filled with class files are not open source. The fact that they are derived from open source is applicable only to what we allow projects to be dependent upon, not what we vote on as a release package. Release votes are on verified open source artifacts. Binary packages are separate from source packages. One cannot vote to approve a release containing a mix of source and binary code because the binary is not open source and cannot be verified to be safe for release (even if it was derived from open source). I thought that was frigging obvious. Why do I need to write documentation to explain something that is fundamental to the open source definition? Also, let's make a clear distinction between binary distributions (i.e. the packages that result from building one of our source releases) and binary dependencies (i.e. binary distributions of upstream projects). AFAICT there's clear consensus that binary distributions are *not* official Apache releases, and we've been pretty consistent about that so far. However, the word on binary dependencies is much less clear. There's explicit Apache policy (category B, etc.) that talks about binary dependencies and plenty of Apache releases contain them. This is clearly not an area where we have consensus. Please point those packages out to me and I will ask Joe to give me root access again so that I can go through and personally delete them from our dist directories. Seriously. I am so tired of having to send these emails, write the documentation, and then watch Java projects to do the wrong things again and again. It is time for the sledgehammer. The Category B is about products in general, not just source packages. Yes, that documentation sucks. Yes, I said so at the time. No, it is NOT an approved policy document of the ASF. The categories are about software licensing of the product as a whole, including hard dependencies and built packages, not whether something is included in a source code package. On Tue, Mar 27, 2012 at 11:50 AM, Roy T. Fielding field...@gbiv.com wrote: Likewise for jar files of dependencies -- they are NOT our product and they MUST NOT be present in the source code package that is voted on for release. Citation needed. Note that the source materials reference you mentioned earlier is vague. It covers stuff like configure scripts in httpd releases, test files, and indeed (as far as it so far has been interpreted) binary dependencies of upstream open source projects. I'm fine if this point needs to be clarified and some current practice terminated, but let's follow standard process to do so. It isn't even remotely vague. Source has only one definition. And it isn't just that document: http://incubator.apache.org/guides/releasemanagement.html#best-practice If any ASF member is aware of an Apache release package that is not 100% open source code, you are hereby instructed to DELETE it from our servers. What hat are you holding here? Which packages explicitly are you referring to? My hat. I'll make it a board directive at the next meeting, if you like. If I had known of any such packages, I would have deleted them already. Don't you remember the similar conversation we had about Jackrabbit releases? The only jar files we should have in subversion are the non-product trees -- the places where we store build tools, the artifacts
Re: Binary dependencies in source releases (Was: [VOTE] Release ManifoldCF 0.5-incubating, RC0)
On Tue, Mar 27, 2012 at 12:57 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: Let's decouple this thread from the specific issue of the ManifoldCF release. There's a long tradition of Apache releases like the ones ManifoldCF is producing, so turning this suddenly into a blocker is IMHO bad business, especially since no legal issues are involved (this is about Apache policy). If we do come to the consensus that releases like this are contrary to Apache policy, *jaw drops*. Huh? But, but, but. But! It's called open *source*. I am honestly just really surprised that this can even come up as a discussion topic. It's such a core concept that source releases are, well, *source* releases, everything is built on top of it. It's not a policy thing, it's a definition thing. The one case I can imagine that might be sort-of ok is if you ship a release with an ant-based build that includes a custom task, and in the source release of your entire project you *also* include a binary version of the custom task, so lazy people (or those without a working bash, or whatever) can avoid running your bootstrap script. (and simlarly, s/ant/maven/, s/task/plugin/, etc). But there's obviously better ways to do that stuff (target that compiles task, taskdef for task in next target, that kind of thing). then affected projects should be given a reasonable amount of time to adapt. Uhm. Normally I'd want to take a similar approach. But. But. But! Open *source*. It's so fundamental to what we do. This seems exactly the reason why we have incubation disclaimers: so we make clear to our downstream users we might be making some mistakes like this, and so that we can then be agile in fixing it. Whoops, made a mistake folks, no downloads here, stand by while the podling makes a new release I'd think that when we find a problem like this after a release is published, we (we as in the PMC, this is not a task to shove onto infra!) should immediately explain the problem and then immediately yank any existing incubation releases that have an issue here. No grace period, no voting, no discussion. Just fix it. That said, I'm not aware of us actually having such a release out there? How to deal with this stuff for TLPs that got it wrong, well, I guess that's a conversation for (a) different mailing list(s). g'night Leo, still trying to pick his jaw up from the floor - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
