So far, when I was thinking of this, my thoughts regarding external libraries were like this: AFAIK, currently, every external library is expected to have a "-license.txt" file attached (with the license of the given library), and the build is then merging them into a single file for the given build output. I think we could enhance this with adding a "-notice.txt" attached to each library and adjusting the build to produce a composite LICENSE and NOTICE files based on data from modules that are being built into the given binary. Hopefully, this can be extended to NBMs as well. I hope that if a convenience binary contains many jars, it is OK to have only one LICENCE and NOTICE at the top, and not in every jar in the distribution (right?).
Jan On Tue, Sep 19, 2017 at 9:02 AM, Ate Douma <a...@douma.nu> wrote: > On 2017-09-19 06:16, Geertjan Wielenga wrote: > >> Many thanks for all these details. Two things jump out at me: >> >> 1. "those NOTICE and LICENSE files *only* refer and apply to what has been >> checked into the (that!) git/subversion source tree, *nothing else*. So no >> references to 3rd party dependencies which are pulled in at build time!" >> >> That's very interesting and it means that the NOTICE and LICENSE will be >> very short and sweet -- i.e., the only license will be Apache, since all >> the source files will be relicensed to Apache. Everything else has been >> removed prior to the donation, i.e., everything that did not belong to >> Oracle, anything that has been licensed to anyone other than Oracle, has >> been removed prior to the donation. Yes, we need to prove that within the >> incubator and document how we prove that. However, I believe based on >> having gone through the donation process from the Oracle side, that the >> only license of the source tree is Apache. >> > > If, and as long as that is the case, then yes. > But if/when files need to be checked in which come with additional > attribution > (notice), or with additional copyright/license requirements, then the root > LICENSE and NOTICE file need to cover those. Common examples I've seen > include > 3rd party images, (java)scripts, etc. which for all practical purposes > need to > at the source level (in git). As long as those have a compatible license, > that > is fine, as long as they are properly marked/referred to in the NOTICE > and/or > LICENSE file(s). > > >> What does this mean for dependencies pulled in at build time? I.e., can >> they then be licensed in any way at all -- i.e., where are the >> rules/guidelines for dependencies pulled in at build time? >> > For dependencies used only *during the build (not > packaged/distributed/released > nor checked in) you simply don't have to worry about. > > >> 2. "NOTICE and LICENSE files *must* be provided in the root of *each* >> package or distribution, or a practical/common equivalent thereof (e.g. in >> a jar-file these should go under META-INF/)" >> >> I had assumed there is one NOTICE and one LICENSE file, in the root of the >> repo. From the above, there could be hundreds, i.e., for each package, it >> would seem. Can you point to somewhere as a reference point for that, >> e.g., >> a project where this is done? It sounds to me like we're going to have >> hundreds of NOTICE and LICENSE files in Apache NetBeans? >> > > Could be :-) But I don't think it will need to be that excessive. > > To clarify one thing first: I've probably put you on the wrong foot with > using > the term 'package'. I did not mean package as in Java package, but more > generically something packaged for distribution, e.g. zip, tar.gz, jar, > war, > nbm, etc. files. > So probably better just ignore the word 'package' in my earlier reply. > > But each and every 'artifact' distributed may indeed need a different > NOTICE > and LICENSE file, but only if it contents require that. > For example for simple jar files, just packaging the compiled java > classes, the > standard NOTICE and LICENSE files are fine. > > And for maven based projects you typically don't need to check in each and > every > NOTICE and LICENSE file for jar or war files but instead have these > added/merged > at build time using the maven-remote-resources-plugin [1], optionally > providing > only the fragment to be appended (under src/main/appended-resources/ME > TA-INF). > > For other/custom distributions, like binary .tar.gz or .nbm files you > might need > to maintain their NOTICE and LICENSE files manually. > > I assume there are ant scripts/tasks which can automate things similarly > like > with maven, but I'm not familiar with ant based projects, so maybe one of > the > other mentors or experienced ASF committers here can chime in. > > A (maven) project I've recently helped mentor through the incubator having > a > fairly complex set of NOTICE and LICENSE file requirements is asterixdb > [2]. > Their git master tree currently contains 9 NOTICE and 14 LICENSE files in > total > if I checked them correctly. > > Ate > > [1] http://maven.apache.org/plugins/maven-remote-resources- > plugin/usage.html > [2] https://github.com/apache/asterixdb > > > >> Thanks, >> >> Geertjan >> >> On Tue, Sep 19, 2017 at 12:17 AM, Ate Douma <a...@douma.nu> wrote: >> >> On 2017-09-16 08:28, Geertjan Wielenga wrote: >>> >>> Hi all, >>>> >>>> Created https://issues.apache.org/jira/browse/NETBEANS-57. >>>> >>>> Trying to find an Apache project to use as reference point, looked >>>> around >>>> a >>>> bit, and Apache Spark seems to be complex enough in terms of its >>>> dependencies to follow: >>>> >>>> https://github.com/apache/spark >>>> >>>> I.e., there's a folder in the above named "licenses": >>>> >>>> https://github.com/apache/spark/tree/master/licenses >>>> >>>> Should such a folder be created in >>>> https://github.com/apache/incubator-netbeans? >>>> >>>> And then, reference those licenses in the LICENSE file: >>>> >>>> https://github.com/apache/spark/blob/master/LICENSE >>>> >>>> And also a NOTICE file should be created, like this one: >>>> >>>> https://github.com/apache/spark/blob/master/NOTICE >>>> >>>> Is that the correct approach and should anything else be considered? >>>> >>>> >>> Warning TL;DR; ahead, but please hang on anyway... >>> >>> I'll offer *my* preferences, and a bit of opinion, but do note that there >>> is no >>> single 'best' or 'perfect' solution, fit for all projects. >>> >>> The ground rules are spelled out and documented at [1] and [2], and IMO >>> should be regarded as the *legal* requirements every solution needs to >>> comply >>> to. >>> >>> [1] http://www.apache.org/legal/release-policy.html#licensing-do >>> cumentation >>> [2] http://apache.org/legal/src-headers.html#notice >>> >>> However, how these ground rules or legal requirements are interpreted and >>> applied differs on detail level for just about every ASF project I've >>> been >>> directly involved with ... >>> Also the common understanding and current expected way how this should or >>> must >>> be done has changed (or progressed if you will) over the years as well. >>> >>> Bottom line is: the IPMC (Incubator PMC) will have to validate and agree >>> with >>> the chosen solution to accept a release candidate, or not. Or may give >>> some >>> initial lenience for acceptable first-time mistakes, to be fixed in >>> future >>> releases. >>> Again: there is no one-fits-all solution, when in doubt do ask on >>> gene...@incubator.apache.org. >>> And preferably *before* starting a vote for a release candidate to reduce >>> the >>> number of times a release candidate is voted down (which AFAICR almost >>> *always* >>> happens the first one or few time...). >>> >>> Furthermore, please accept my personal disclaimer that in most cases I >>> also >>> tend to overlook or miss a few bits and bytes the first few times a new >>> project >>> needs to get its LICENSE and NOTICE file(s) in order. >>> >>> It can and typically will be a tiresome exercise in the beginning, nobody >>> should >>> take this lightly. But once the basic process has been fleshed out and a >>> first >>> complete and verified release has gone through successfully, things will >>> get >>> easier, promise :-) >>> >>> Now concerning the 'correct approach'. >>> I'd like to provide some guidelines or rules of the thumb of my own, as >>> I've >>> been using and advocating over the years, and which worked, and still >>> work, for >>> several ASF projects. >>> But feel free to discuss/question these 'rules', to get a better >>> understanding >>> or possibly (hopefully) improve/fix/refine them as needed. >>> In the end, its the Netbeans PPMC, and during incubation ultimately the >>> IPMC, >>> which needs to decide and accept the approach taken. >>> >>> >>> a) NOTICE and LICENSE files *only* apply to what is contained within a >>> 'package' >>> or distribution. They do NOT and should NOT refer to or address anything >>> NOT >>> within that package or distribution. >>> >>> b) NOTICE and LICENSE files *must* be provided in the root of *each* >>> package or >>> distribution, or a practical/common equivalent thereof (e.g. in a >>> jar-file >>> these >>> should go under META-INF/) >>> >>> c) The ASF git (or likewise subversion) repository should also be >>> regarded >>> as a >>> 'distribution' of its own. It therefore also requires a NOTICE and >>> LICENSE >>> file >>> checked in the root folder of the git tree. >>> Per rule a) those NOTICE and LICENSE files *only* refer and apply to what >>> has >>> been checked into the (that!) git/subversion source tree, *nothing else*. >>> So no references to 3rd party dependencies which are pulled in at build >>> time! >>> >>> d) A *source* release typically includes everything from the (that) git >>> source >>> tree, in which case the git tree root NOTICE and LICENSE files are 'good >>> to go'. >>> But it also may (also) combine multiple git repositories, or split them >>> up. >>> For example a common approach is to also provide (distribute) one or more >>> 'convenient' <module>-sources.jar files. In which case possibly only a >>> subset of >>> the 'root' NOTICE and LICENSE file is needed/should be bundled. >>> >>> e) A binary 'convenience' distribution (which is NOT the release, and >>> only >>> may >>> be distributed *together* with the corresponding source release), >>> typically >>> also bundles 3rd party dependencies. The NOTICE and LICENSE file packaged >>> with >>> such binary distribution then also MUST cover those 3rd party >>> dependencies, see >>> also a) above. Which means *different* (extended) NOTICE and LICENSE >>> files >>> will >>> be needed, compared to the source release. >>> For Netbeans I think this may apply to nbm files if I'm correct. And each >>> nbm >>> file may even require a different/dedicated NOTICE and LICENSE file, >>> depending >>> on its content. >>> Also notable is how to handle ASF, or for that matter, any 3rd party AL >>> 2.0 >>> based dependencies: while their LICENSE is the same (AL 2.0), if these >>> 3rd party dependencies carry a NOTICE file (which every ASF project is >>> required >>> to do), any *applicable* content from those NOTICE files needs to carried >>> (merged) into your own NOTICE file as well (AL 2.0, 4.4d). >>> This also explains why NOTICE files should *only* contain notices which >>> are >>> (legally) required, nothing more. >>> Do NOT put nice or friendly attributions ('thank you' or whatnot) in a >>> NOTICE >>> file, unless required. Friendly attributions may/should go in a non-legal >>> README >>> or something similar instead! >>> >>> f) All NOTICE attributes can/must only go into the one NOTICE file. >>> 3rd party LICENSEs may be appended to the main LICENSE file, or >>> referred/linked >>> to, as long as they are bundled within the distribution. Providing only >>> remote linking (e.g. using an url) is not allowed. >>> >>> g) Finally (and in practice this is often a nasty/tricky one): don't >>> blindly >>> trust or rely on 3rd party dependencies bundling (or not) 'a' LICENSE >>> (and/or NOTICE) file. While at the ASF we take these things very >>> seriously >>> (and still >>> often make mistakes...) its not uncommon (or rather common in my >>> experience) >>> that other projects don't. >>> Either there is no LICENSE/NOTICE bundled, or it is clearly wrong, >>> outdated, incomplete or contains 'everything from the kitchen sink'. >>> We (the PMC, ultimately responsible for voting and accepting a release) >>> need to >>> do the proper thing and verify or hunt down the actual/current LICENSE >>> and/or >>> NOTICEs applicable to 3rd party dependencies. For as much >>> reasonable/feasible. >>> But often we can reuse verification/validation already done by other ASF >>> projects, so for common dependencies it usually isn't that bad. >>> >>> That's it :-) >>> >>> Note therefore that the above example NOTICE and LICENSE files from >>> Apache >>> Spark >>> 'violate' the above guidelines on a) and c). >>> Which isn't legally incorrect, but unnecessary and unwanted. It is making >>> it >>> more difficult for downstream consumers to figure out which licenses >>> 'really' >>> apply, or even downright annoying as per AL 2.0 section 4.4d) the whole >>> of >>> the >>> NOTICE file content *must* be copied by downstream re-distributers... >>> >>> HTH, >>> Ate >>> >>> >>> >>> The SGA received from Oracle provides a starting point, i.e., a list of >>>> 3rd >>>> party references, which I propose taking as the first step, i.e., for >>>> the >>>> initial draft of the LICENSE and NOTICE, after which we will document >>>> the >>>> process by which we make sure that these are accurate. >>>> >>>> Thanks, >>>> >>>> Gj >>>> >>>> >>>> >>> >> >
