Legal question asked at https://issues.apache.org/jira/browse/LEGAL-601
Kenn On Fri, Feb 4, 2022 at 7:58 AM Danny McCormick <[email protected]> wrote: > Sure - I'm happy to help out with the Actions setup (and/or with the Go > template). I will say though, the Actions config should be pretty darn > simple for these examples - > https://github.com/davidcavazos/beam-java/blob/main/.github/workflows/test.yaml > seems right, for each language configuration we're targeting we basically > just want a job with: > > - checkout > - setup-<language> > - inlined script to run tests > > Always happy to help with or consult on any actions issues 🙂 > > Thanks, > Danny > > On Fri, Feb 4, 2022 at 10:21 AM Kerry Donny-Clark <[email protected]> > wrote: > >> Danny has extensive experience with GitHub actions, and may be able to >> help out. >> Kerry >> >> On Thu, Feb 3, 2022, 11:47 PM Kenneth Knowles <[email protected]> wrote: >> >>> I'm convinced on all points. My main motivation was to keep it simple. >>> But of course we should keep it simple for users, not us :-) >>> >>> I can take on the task of asking about MIT license and requesting the >>> repos be created. Not sure if it needs my level of privileges but I'm happy >>> to do it anyhow. >>> >>> Kenn >>> >>> On Wed, Feb 2, 2022 at 10:30 AM Robert Bradshaw <[email protected]> >>> wrote: >>> >>>> On Wed, Feb 2, 2022 at 10:12 AM David Cavazos <[email protected]> >>>> wrote: >>>> > >>>> > MIT is much more permissive, but I also don't have any problems >>>> changing it to Apache license. In any case, how about we create the >>>> following repos? >>>> >>>> For these starter projects, we don't want to encumber any users of >>>> these templates with any particular licensing requirements (right?) >>>> and we don't even care about attribution. We want these to be pretty >>>> much as close to public domain as possible. That's not what the Apache >>>> licence does. (If it's even relevant, a good argument could likely be >>>> made for de minis or fair use, but I think it's best to be explicit >>>> about this. Perhaps this'd be a good question for apache legal? >>>> >>>> > apache/beam-starter-java >>>> > apache/beam-starter-python >>>> > apache/beam-starter-go >>>> > apache/beam-starter-kotlin >>>> > apache/beam-starter-scala >>>> > >>>> > We'll start by populating the Java one which is the most pressing one >>>> and the one that is ready, but the rest should be simpler. >>>> > >>>> > +David Huntsperger, tldr; these are minimal starter projects for >>>> every language. Once we have Java, Python and Go, it might be a good idea >>>> to change the quickstarts to use these instead of the word count. There is >>>> already a dedicated word count walkthrough so I think that is already >>>> covered. >>>> > >>>> > If we all agree on the repo names, who can help us create them? >>>> > >>>> > On Thu, Jan 27, 2022 at 12:58 PM Robert Bradshaw <[email protected]> >>>> wrote: >>>> >> >>>> >> On Tue, Jan 18, 2022 at 6:17 AM Kenneth Knowles <[email protected]> >>>> wrote: >>>> >> > >>>> >> > Agree with Luke here. "Just git clone and go" is a big part of it. >>>> >> > >>>> >> > But also the answer to "I simply don't know what one would put in >>>> a Python repo than, other than a bare setup.py that lists a dependency on >>>> apache_beam" is answered by David's initial email and his repo, namely: >>>> >> > >>>> >> > - GitHub Actions configuration >>>> >> > - README.md >>>> >> > - example that already runs >>>> >> >>>> >> OK, fair enough. >>>> >> >>>> >> > - LICENSE (notably you've got it as MIT but to be part of Apache >>>> software it needs to be ASL2) >>>> >> >>>> >> On the topic of licence, it's a bit tricky because one doesn't want >>>> to >>>> >> bind the users of such a template as being a derivative work of a >>>> >> too-restrictive licence. The licence of the template itself should >>>> >> generally be very permissive. >>>> >> >>>> >> > On Fri, Jan 14, 2022 at 2:34 PM Luke Cwik <[email protected]> >>>> wrote: >>>> >> >> >>>> >> >> I think for consistency it makes sense to users to be told to >>>> checkout this git repo for the language of your choice and run. Some repos >>>> will have more/less than others when it comes to setup necessary. >>>> >> >> >>>> >> >> On Fri, Jan 14, 2022 at 2:26 PM Robert Bradshaw < >>>> [email protected]> wrote: >>>> >> >>> >>>> >> >>> +1 for doing this for Java, as setting up a project there is >>>> quite >>>> >> >>> complicated. I simply don't know what one would put in a Python >>>> repo >>>> >> >>> than, other than a bare setup.py that lists a dependency on >>>> >> >>> apache_beam. We don't have recommendations on file layout, etc. >>>> more >>>> >> >>> than that (though there's plenty of generic advice to be found >>>> out >>>> >> >>> there on the topic). I have a hunch go is similar, and javascript >>>> >> >>> would be as well (npm install apache-beam and your package.json >>>> file >>>> >> >>> gets updated). >>>> >> >>> >>>> >> >>> On Fri, Jan 14, 2022 at 2:17 PM Luke Cwik <[email protected]> >>>> wrote: >>>> >> >>> > >>>> >> >>> > There are several examples already within the Beam repo found >>>> in: >>>> >> >>> > https://github.com/apache/beam/tree/master/examples >>>> >> >>> > https://github.com/apache/beam/tree/master/sdks/go/examples >>>> >> >>> > >>>> https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples >>>> >> >>> > >>>> >> >>> > >>>> >> >>> > On Fri, Jan 14, 2022 at 11:07 AM Sachin Agarwal < >>>> [email protected]> wrote: >>>> >> >>> >> >>>> >> >>> >> I'd love to do something other than Wordcount just for >>>> novelty/freshness but agreed with the suggestion that having an example in >>>> each quickstart would be ideal. >>>> >> >>> >> >>>> >> >>> >> On Fri, Jan 14, 2022 at 11:06 AM David Huntsperger < >>>> [email protected]> wrote: >>>> >> >>> >>> >>>> >> >>> >>> + 1 to a separate repo for each language. >>>> >> >>> >>> >>>> >> >>> >>> Would it make sense to include the Wordcount example in each >>>> repo? I know that makes the repos less minimal, but we could rewrite the >>>> quickstarts around these repos instead of the current Wordcount examples. >>>> Or maybe we don't need to use the Wordcount example in the quickstarts... >>>> >> >>> >>> >>>> >> >>> >>> On Wed, Jan 12, 2022 at 1:54 PM David Cavazos < >>>> [email protected]> wrote: >>>> >> >>> >>>> >>>> >> >>> >>>> I agree with dropping the archetypes. Less maintenance is >>>> preferable, and the github repos are more flexible and maintainable. >>>> >> >>> >>>> >>>> >> >>> >>>> How about we create: >>>> >> >>> >>>> >>>> >> >>> >>>> apache/beam-starter-java >>>> >> >>> >>>> apache/beam-starter-python >>>> >> >>> >>>> apache/beam-starter-go >>>> >> >>> >>>> >>>> >> >>> >>>> During our OKR planning, +Keith Malvetti would prefer >>>> having repos for all languages. It makes sense for consistency as well. >>>> >> >>> >>>> >>>> >> >>> >>>> On Mon, Jan 10, 2022 at 5:14 PM Luke Cwik <[email protected]> >>>> wrote: >>>> >> >>> >>>>> >>>> >> >>> >>>>> As long as we have tags so that people can pull out a >>>> specific version of the examples that coincides with a specific SDK version >>>> then we could drop the archetypes. >>>> >> >>> >>>>> >>>> >> >>> >>>>> On Mon, Jan 10, 2022 at 4:09 PM Brian Hulette < >>>> [email protected]> wrote: >>>> >> >>> >>>>>> >>>> >> >>> >>>>>> > Being such minimal examples, I don't expect them to >>>> break commonly, but I think it would be good to make sure tests aren't >>>> failing when a release is published. >>>> >> >>> >>>>>> >>>> >> >>> >>>>>> Yeah it would be very unfortunate if we discovered a >>>> breakage after the release. Agree we should verify RCs (document as part of >>>> the release process), or even better, add automation to verify the repo >>>> against snapshots. The automation could be nice to have anyway since it >>>> provides an example for users to follow if they want to test against >>>> snapshots and report issues to us sooner. >>>> >> >>> >>>>>> >>>> >> >>> >>>>>> >>>> >> >>> >>>>>> If we move forward with this can we drop the archetype? >>>> >> >>> >>>>>> >>>> >> >>> >>>>>> On Fri, Jan 7, 2022 at 3:54 PM Luke Cwik < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>> >>>> >> >>> >>>>>>> Sounds reasonable. >>>> >> >>> >>>>>>> >>>> >> >>> >>>>>>> On Wed, Jan 5, 2022 at 12:47 PM David Cavazos < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>>> >>>> >> >>> >>>>>>>> I personally like the idea of a separate repo since we >>>> can see how a true minimal project looks like. Having it in the main repo >>>> would inherit build file configurations and other settings that would be >>>> different from a clean project, so it could be non-trivial to adapt. Also >>>> as its own repo, it's easier to clone and modify, or create an instance of >>>> the template. >>>> >> >>> >>>>>>>> >>>> >> >>> >>>>>>>> Dependabot can take care of updating the Beam version >>>> and other dependencies automatically. Testing is already set up via GitHub >>>> actions for every pull request, so it would automatically be tested as soon >>>> as there is a new dependency version available. >>>> >> >>> >>>>>>>> >>>> >> >>> >>>>>>>> Being such minimal examples, I don't expect them to >>>> break commonly, but I think it would be good to make sure tests aren't >>>> failing when a release is published. >>>> >> >>> >>>>>>>> >>>> >> >>> >>>>>>>> I'm okay with having one repo per language, and having >>>> all the build systems we want to support for them. As long as we document >>>> which files are for which build system. That way there are less repos to >>>> maintain. >>>> >> >>> >>>>>>>> >>>> >> >>> >>>>>>>> On Mon, Dec 13, 2021 at 9:25 AM Luke Cwik < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>>>> >>>> >> >>> >>>>>>>>> The github repo is definitely more flexible then the >>>> archetypes but the archetypes have a few conveniences since they are >>>> integrated with apache/beam repo. For example, updates/testing are done at >>>> the same time a corresponding change to the main repo is done (like library >>>> version updates), they are released when the SDK is released. >>>> >> >>> >>>>>>>>> >>>> >> >>> >>>>>>>>> Should these be part of the main repo, or a single >>>> starter repo containing all the starters or one per language or one per >>>> build system? >>>> >> >>> >>>>>>>>> >>>> >> >>> >>>>>>>>> When should updates to the starter happen? >>>> >> >>> >>>>>>>>> How as a community do we get them to happen (e.g. >>>> release manager owns it)? >>>> >> >>> >>>>>>>>> >>>> >> >>> >>>>>>>>> >>>> >> >>> >>>>>>>>> On Sun, Dec 12, 2021 at 4:06 PM David Cavazos < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>>>>> >>>> >> >>> >>>>>>>>>> We could do the Maven archetype, but that wouldn't >>>> work very well for Gradle and SBT users. I think a GitHub template might be >>>> the more flexible option, and we could have something similar for other >>>> languages as well. Having said that, we could still create a Maven >>>> archetype. If someone is familiar with that process, please let me know >>>> since I'm not too familiar with Maven and its ecosystem. >>>> >> >>> >>>>>>>>>> >>>> >> >>> >>>>>>>>>> @Ahmet Altay I think right now we only need to pin >>>> down the name of the repo, create it, and move the code there. I was >>>> thinking either `apache/beam-java-template` or `apache/beam-java-starter`. >>>> What do you think? >>>> >> >>> >>>>>>>>>> >>>> >> >>> >>>>>>>>>> What would be the next steps on creating the repo? >>>> >> >>> >>>>>>>>>> >>>> >> >>> >>>>>>>>>> On Thu, Dec 9, 2021 at 11:09 AM Ahmet Altay < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>>>>>> >>>> >> >>> >>>>>>>>>>> This is great David. Was there any progress on this? >>>> Do you need help? >>>> >> >>> >>>>>>>>>>> >>>> >> >>> >>>>>>>>>>> On Wed, Dec 1, 2021 at 3:54 PM Brian Hulette < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>> This is cool, thanks! >>>> >> >>> >>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>> We do have a template in apache/beam already, built >>>> with Maven Archetype [1]. It's what powers the Java quickstart [2]. Could >>>> we de-dupe these (e.g. reference the GitHub template in the quickstart, or >>>> co-locate the archetype with the GitHub template)? >>>> >> >>> >>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>> As far as creating an Apache repo, would we put >>>> this somewhere like apache/beam-java-template? I think apache repositories >>>> like beam-* are allowed. >>>> >> >>> >>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>> Brian >>>> >> >>> >>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>> [1] https://maven.apache.org/archetype/index.html >>>> >> >>> >>>>>>>>>>>> [2] >>>> https://beam.apache.org/get-started/quickstart-java/#get-the-example-code >>>> >> >>> >>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>> On Wed, Dec 1, 2021 at 11:30 AM David Cavazos < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>> +Ahmet Altay >>>> >> >>> >>>>>>>>>>>>> +Valentyn Tymofieiev >>>> >> >>> >>>>>>>>>>>>> +Kenneth Knowles >>>> >> >>> >>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>> Please feel free to include anyone else! >>>> >> >>> >>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>> On Mon, Oct 25, 2021 at 11:31 AM David Cavazos < >>>> [email protected]> wrote: >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> Hi Beam community! >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> To make it easier to create a new Beam Java >>>> project, I've been working on a GitHub template containing a minimal Beam >>>> Java pipeline for people to start with. >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> Link to the GitHub template: >>>> https://github.com/davidcavazos/beam-java >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> So far, here's what the template contains: >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> Minimal "Hello World" Beam pipeline >>>> >> >>> >>>>>>>>>>>>>> Minimal test file >>>> >> >>> >>>>>>>>>>>>>> Build files for Gradle, sbt, and Maven (Direct >>>> runner) >>>> >> >>> >>>>>>>>>>>>>> Continuous integration via GitHub actions (around >>>> 1-2 minutes to run) >>>> >> >>> >>>>>>>>>>>>>> README with instructions on how to build, run, >>>> test, and add other runners >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> It's easy to create a new GitHub repo from a >>>> template. >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> Next steps >>>> >> >>> >>>>>>>>>>>>>> >>>> >> >>> >>>>>>>>>>>>>> Some reviewers to make sure everyone is happy >>>> with it 🙂 >>>> >> >>> >>>>>>>>>>>>>> Right now it lives in my personal GitHub account, >>>> so we need to create an Apache repo to host it >>>> >> >>> >>>>>>>>>>>>>> Update/create docs with instructions on how to >>>> create a new Beam Java pipeline >>>> >>>
