The owners at pkg go.dev say they can't properly recognize the python license (see https://github.com/golang/go/issues/45095) due to the license being somewhat domineering (a go project could *only* have that license if it had that license, apparently).
We also can't do a directory specific solution for the Go code. Pkg.go.dev is an automated crawler and not human curated so they need to be more conservative (all general LICENSE files to the root of the repo are included) One suggestion is that instead of appending the license to a single file, we could separate that license into it's own LICENSE.python file which wouldn't be picked up by the tooling. I'll take a look at our various license handling /publishing code too and make appropriate changes to make sure it's not lost in those cases. Another alternative that might work (I've asked in the issue) is maintaining the license in a LICENSE file, but moving that content in particular to the sdks/python directory. But this is less preferable since we have python code in more places than that. My vote is for LICENSE.python, and if there are no objections by the next release cut I'll put forth the PR. I'd rather not make the change so close to it, and we can cherry pick it otherwise. Robert Burke Beam Go Busybody On Wed, Mar 24, 2021, 9:25 AM Robert Burke <[email protected]> wrote: > I'm less concerned about the Go Doc at this point. > > 0. The Go SDK is still experimental (I know I know, almost there), making > it less severe of an issue. > 1 The interpretation pkg.go.dev does is live with the release. > 1.a That is, if they start accepting the python 2.0 license blob that's > fixed for all affected versions. > 1.b Further, if we do cherrypick licence textual fixes into the older > versions for the licence matching, then the site will pick those up again. > 2. Users can run the godoc tool > <https://pkg.go.dev/golang.org/x/tools/cmd/godoc> themselves to get > package documentation if they want the docs of the version they're using, > or simply have the programming language server (for go, GoPLS) lift that > into their IDEs as needed. > > > On Mon, 22 Mar 2021 at 16:42, Ahmet Altay <[email protected]> wrote: > >> For visibility: Change is cherry picked to 2.29.0 release branch ( >> https://github.com/apache/beam/pull/14304). >> >> On Mon, Mar 22, 2021 at 12:37 PM Kenneth Knowles <[email protected]> wrote: >> >>> So if I understand correctly, the options for a correct license in the >>> released artifacts are: >>> >>> - revert the change >>> - build some automation for bundled jars >>> - do something manual? >>> >>> Kenn >>> >>> On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <[email protected]> >>> wrote: >>> >>>> Pros/cons for a cherrypick: >>>> (+) Fixes regression for licenses in released Java artifacts. >>>> (-) It's possible it will permanently break docs on pkg.go.dev for >>>> 2.29.0 if https://github.com/golang/go/issues/45095 requires changes >>>> on our end (e.g. fixing the PSF License text). >>>> >>>> My sense is the pro outweighs the con here, but I could be convinced >>>> otherwise. I guess that makes me +0 for cherrypick. >>>> >>>> Brian >>>> >>>> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <[email protected]> >>>>> wrote: >>>>> >>>>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick? >>>>>> >>>>> >>>>> I do not think so. I have not filed a jira or started a cherry pick >>>>> pr. >>>>> >>>>> Sorry, I was not sure if we agreed to cherry pick or not. Do you want >>>>> me to do that? >>>>> >>>>> >>>>>> >>>>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> I also noticed (with a help of an automated tool) that >>>>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES >>>>>>> includes additional licenses not included in >>>>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI >>>>>>> since Dataflow runner is released as a separate jar artifact, and the >>>>>>> licenses in question (GPL 2.0, CDDL) pertain to its dependencies, or we >>>>>>> need to include those licenses as well? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev >>>>>>>>> >>>>>>>>> > and probably cherry pick it into the affected release branches. >>>>>>>>> Even if we do this, the Java artifacts from the affected releases >>>>>>>>> are missing the additional LICENSE text. >>>>>>>>> >>>>>>>> >>>>>>>> IMO we can skip the cherry picks perhaps with the exception of the >>>>>>>> upcoming 2.29 release. >>>>>>>> >>>>>>>>> >>>>>>>>> > I do not know how to interpret this ASF guide. As an example >>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, >>>>>>>>> and a >>>>>>>>> licenses directory. There are even overlapping mentions. >>>>>>>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's >>>>>>>>> guide that presumably targets engineers is not particularly clear to >>>>>>>>> me. >>>>>>>>> This was just my tenuous understanding after a quick review. >>>>>>>>> >>>>>>>> >>>>>>>> Agreed. We can ask LEGAL for further clarification. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Thank you Rebo. I agree with reverting first and then figure out >>>>>>>>>> the next steps. >>>>>>>>>> >>>>>>>>>> Here is a PR to revert your change: >>>>>>>>>> https://github.com/apache/beam/pull/14267 >>>>>>>>>> >>>>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Looking at the history it seems that before the python text was >>>>>>>>>>> added, pkg.go.dev can parse the license stack just fine. It >>>>>>>>>>> doesn't recognize the PSF license, and fails closed entirely as a >>>>>>>>>>> result. >>>>>>>>>>> >>>>>>>>>>> I've filed an issue with pkg.go.dev ( >>>>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is >>>>>>>>>>> fixed, the affected versions will become visible as well. >>>>>>>>>>> >>>>>>>>>>> In the meantime, we should revert my change which clobbered the >>>>>>>>>>> other licenses and probably cherry pick it into the affected release >>>>>>>>>>> branches. >>>>>>>>>>> >>>>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing >>>>>>>>>>> but python can use it and call it the PSF license. However it is a >>>>>>>>>>> redistribution friendly license, which is what matters. >>>>>>>>>>> >>>>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thank you for this email. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I just noticed that there was a recent change to our LICENSE >>>>>>>>>>>>> file to make it exactly match the Apache 2.0 License [1]. This >>>>>>>>>>>>> seems to be >>>>>>>>>>>>> the result of two conflicting LICENSE issues. >>>>>>>>>>>>> >>>>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy >>>>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our >>>>>>>>>>>>> documentation didn't show up there [3]. >>>>>>>>>>>>> >>>>>>>>>>>>> Java artifact LICENSE issue: The removed text contained >>>>>>>>>>>>> information relevant to "convenience binary distributions". This >>>>>>>>>>>>> text was >>>>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we >>>>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. >>>>>>>>>>>>> The >>>>>>>>>>>>> suggested solution (that we went with) was to just add the >>>>>>>>>>>>> information to >>>>>>>>>>>>> the root (source) LICENSE. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Python distribution is missing both files as well. ( >>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not sure that that solution is consistent with this ASF >>>>>>>>>>>>> guide [6] which states: >>>>>>>>>>>>> >>>>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the >>>>>>>>>>>>> contents of the distribution they reside in. Only components and >>>>>>>>>>>>> resources >>>>>>>>>>>>> that are actually included in a distribution have any bearing on >>>>>>>>>>>>> the >>>>>>>>>>>>> content of that distribution's NOTICE and LICENSE. >>>>>>>>>>>>> >>>>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for >>>>>>>>>>>>> our root (source) LICENSE, and the correct way to deal with binary >>>>>>>>>>>>> artifacts is to generate per-artifact LICENSE/NOTICE files. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I do not know how to interpret this ASF guide. As an example >>>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE >>>>>>>>>>>> file, and a >>>>>>>>>>>> licenses directory. There are even overlapping mentions. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact >>>>>>>>>>>>> issue has regressed. I can think of two potential solutions to >>>>>>>>>>>>> resolve both: >>>>>>>>>>>>> 1) Restore the "convenience binary distributions" information, >>>>>>>>>>>>> and see if we can get pkg.go.dev to allow it. >>>>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for >>>>>>>>>>>>> Java binary artifacts. >>>>>>>>>>>>> >>>>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more >>>>>>>>>>>>> tenable, but less correct since it's adding information not >>>>>>>>>>>>> relevant to the >>>>>>>>>>>>> source release. >>>>>>>>>>>>> >>>>>>>>>>>>> Brian >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657 >>>>>>>>>>>>> [2] https://pkg.go.dev/license-policy >>>>>>>>>>>>> [3] >>>>>>>>>>>>> https://pkg.go.dev/github.com/apache/[email protected]+incompatible/sdks/go/pkg/beam >>>>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461 >>>>>>>>>>>>> [5] >>>>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E >>>>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.htmlI'll take a >>>>>>>>>>>>> look at our various license handling /publishing code too and make >>>>>>>>>>>>> appropriate changes to make sure it's not lost in those cases. >>>>>>>>>>>>> >>>>>>>>>>>>
