I thought that the purpose of this discussion was to find the one or two platforms for Heron to support in order to keep down the complexity of making the first binary release for the community.
Other platforms could be on a roadmap for future releases. Regards, Dave > On May 26, 2020, at 12:26 PM, Ning Wang <wangnin...@gmail.com> wrote: > > My concern about "latest" is that I feel most people hold it for some time > before migrating to the latest version (if it ever happens). If there are > numbers, it could make sense to use the most popular version (plus the > other criterias). > > On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <oli...@oliverbristow.co.uk> > wrote: > >> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but >> would the latest LTS be a better choice? 18.04 was released before Debian >> 10, so that may be an argument for 20.04 re. compatibility if Debian 10 is >> the standard. >> >> There's already a 20.04 Dockerfile + I It looks like the vagrant setup will >> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and >> seems to run fine so far. >> >> On Tue, 26 May 2020 at 19:30, Josh Fischer <j...@joshfischer.io> wrote: >> >>> Thanks for further clarifying. Yes, that is correct. >>> >>> >>> >>> On Tue, May 26, 2020 at 1:16 PM Ning Wang <wangnin...@gmail.com> wrote: >>> >>>> +1. >>>> >>>> So my understanding is: >>>> binary (in future release): Debian 10. >>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be >>>> updated in the future after more discussions/votes. >>>> >>>> others will be moved to a "extra" folder only for reference. >>>> >>>> >>>> >>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer <j...@joshfischer.io> >>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Based on what I've read on this thread a combination of CentOS/Debian >>>> would >>>>> be the better options for us to support multiple versions for the >>>>> community. We will also support one version of Ubuntu. >>>>> >>>>> So..... I will start a vote in 72 hours that calls on the vote on >> this. >>>>> I'm going to write down my understanding of what this thread covers >> to >>>> get >>>>> some actionable topics to vote on. If I've written anything down >>>>> incorrectly please speak up before we start a vote. >>>>> >>>>> Debian: >>>>> - Debian10 is the official docker image to support and be >> published >>>> to >>>>> Docker Hub, we will also support Debian9. >>>>> CentOS: >>>>> - We will support CentOS7. Note, I do not think it has native >>>>> Python3.7 support, but we are also not there yet. >>>>> Ubuntu: >>>>> - We will still support one version. At this time the version is >>>> 18.04. >>>>> >>>>> We will upgrade supported versions of each OS as updates/deprecation >> of >>>>> support are needed. >>>>> >>>>> Again let's wait 72 hours before starting a vote. Also please check >>> that >>>>> the statements I wrote down are what you understood! >>>>> >>>>> See below for notes on lazy consensus: >>>>> https://community.apache.org/committers/lazyConsensus.html >>>>> >>>>> - Josh >>>>> >>>>> >>>>> On Wed, May 20, 2020 at 9:41 PM Ning Wang <wangnin...@gmail.com> >>> wrote: >>>>> >>>>>> +1 >>>>>> >>>>>> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis < >>>> nicholas.ne...@gmail.com >>>>>> >>>>>> wrote: >>>>>> >>>>>>> I like those options. >>>>>>> >>>>>>> On Wed, May 20, 2020 at 10:15 PM Josh Fischer < >> j...@joshfischer.io >>>> >>>>>> wrote: >>>>>>> >>>>>>>> Let's wait 3 more days for anyone to give input. Then we can >>>> start a >>>>>>> vote >>>>>>>> to make a final decision on which containers for us to support. >>>> Keep >>>>>> in >>>>>>>> mind that even though we do need to plan for Python3 support >>> (There >>>>> is >>>>>>> work >>>>>>>> in progress now) we do not need it as of yet. >>>>>>>> >>>>>>>> - Josh >>>>>>>> >>>>>>>> On Wed, May 20, 2020 at 8:34 PM thinker0 <think...@gmail.com> >>>> wrote: >>>>>>>> >>>>>>>>> My experience is mainly for CentOS 7, so I think it would be >>> nice >>>>> if >>>>>> it >>>>>>>> was >>>>>>>>> included. >>>>>>>>> >>>>>>>>> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <j...@joshfischer.io >>> 님이 >>>> 작성: >>>>>>>>> >>>>>>>>>> Well thought out Windham. I like how you narrowed the >>> criteria >>>>>> down >>>>>>> to >>>>>>>>>> Debian10/CentOS7(depending on PY3 support) thinking about >>>> overall >>>>>>>> support >>>>>>>>>> and what others would need/want. >>>>>>>>>> >>>>>>>>>> +1 >>>>>>>>>> >>>>>>>>>> - Josh >>>>>>>>>> >>>>>>>>>> On Mon, May 18, 2020 at 8:39 PM Windham Wong < >>>>>>> windham.w...@stormeye.io >>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> My experience is that, people tends to pick their >> container >>>>> image >>>>>>> OS >>>>>>>>>> based >>>>>>>>>>> on few criterias only: >>>>>>>>>>> 1) image size, alpine always win, then debian/centos and >>> 3rd >>>> is >>>>>>>>>>> ubuntu/centos. alpine is always the smallest but not much >>>>> library >>>>>>>>>> provided >>>>>>>>>>> and require to compile manually. debian is always good >>>> because >>>>> it >>>>>>> has >>>>>>>>>>> apt-get to provide loads of library. centos is cool as >>> debian >>>>>>> because >>>>>>>>> it >>>>>>>>>>> also has good yum repo. ubuntu is always the last because >>> it >>>> is >>>>>>> fat, >>>>>>>>> but >>>>>>>>>> it >>>>>>>>>>> provides very large range of libraries to use. >>>>>>>>>>> 2) kernel feature, as mentioned last email, ubuntu has >>> issue >>>>> with >>>>>>> DNS >>>>>>>>> and >>>>>>>>>>> some sort of network discover issues. I remember alpine >> has >>>>> some >>>>>> as >>>>>>>>> well, >>>>>>>>>>> because it is very plain, requires many fine tunes be >> done >>>>>>> manually. >>>>>>>> in >>>>>>>>>>> this case, centos and debian should win. >>>>>>>>>>> 3) linux stream, obviously, alpine, debian/ubuntu, >> centos, >>>> are >>>>> 3 >>>>>>>>>> different >>>>>>>>>>> streams that use different native tools. Poeple just pick >>>> their >>>>>>> base >>>>>>>>>> image >>>>>>>>>>> based on what they often use in their life. However, I >> see >>> a >>>>> lot >>>>>>>> debian >>>>>>>>>> and >>>>>>>>>>> alpine on docker hub because they are just smaller in >> size. >>>>>>>>>>> 4. library support. Python library is a very good example >>> in >>>>> this >>>>>>>> case. >>>>>>>>>> As >>>>>>>>>>> Heron is turning to Python 3, which specifically Python >>> 3.7+, >>>>>> some >>>>>>>>>>> discontinued based OS version does not provide the >> support >>>>>> anymore. >>>>>>>> We >>>>>>>>>> have >>>>>>>>>>> tested out that, debian9 supports only up to Python3.5, >>>> ubuntu >>>>>>> 14.04 >>>>>>>>> does >>>>>>>>>>> not support python3 and ubuntu 16.04 supports up to >> Python >>>> 3.5 >>>>>>> only. >>>>>>>>>>> >>>>>>>>>>> I am not a big fan of using centos for docker. Concluding >>>> all 4 >>>>>>>> points >>>>>>>>>>> above, I believe keeping centos7 (if it supports >> python37), >>>>>>> debian10 >>>>>>>>> (not >>>>>>>>>>> debian9), ubuntu 18.04 (supports python3.7+) would be the >>>> best >>>>>>>> solution >>>>>>>>>> at >>>>>>>>>>> the moment. We have to cut off the old images due to our >>>>> library >>>>>>>>> upgrade >>>>>>>>>>> and this is unavoidable. Otherwise, we would have tons of >>>>> issues >>>>>>>>> sending >>>>>>>>>> in >>>>>>>>>>> about library support in the future and the workload >> could >>>> be a >>>>>> lot >>>>>>>>> more >>>>>>>>>>> for just answering questions. >>>>>>>>>>> >>>>>>>>>>> Windham Wong >>>>>>>>>>> Co-Founder, Technical Director of >>>>>>>>>>> Stormeye.io, Hong Kong Managed Security Operation Center >>>>> Limited >>>>>>>>>>> Email // windham.w...@stormeye.io (mailto: >>>>>> windham.w...@stormeye.io >>>>>>> ) >>>>>>>>>>> Phone // +852_3590_2212_|_+852_9832_0707 >> (tel:+85235902212) >>>>>>>>>>> Fax // +852_3590_2202 (tel:+852_3590_2202) >>>>>>>>>>> >>>>>>>>>>> On 5月 19 2020, at 6:09 早上, Josh Fischer < >>> j...@joshfischer.io >>>>> >>>>>>> wrote: >>>>>>>>>>>> Hi All, I think using CentOS and either Debian or >> Ubuntu >>>> is a >>>>>>> good >>>>>>>>>> idea. >>>>>>>>>>> I would pick Debian over Ubuntu for the reason that Nick >>>>> pointed >>>>>>> out >>>>>>>>> in a >>>>>>>>>>> previous email of DNS issues in Kubernetes. I"ve copy and >>>>> pasted >>>>>> a >>>>>>>>>> section >>>>>>>>>>> of Nick's previous email. ### Start I've run into issues >>> with >>>>>>> Ubuntu >>>>>>>>>> images >>>>>>>>>>> having DNS issues in Kubernetes. >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 ### End >>> As >>>>> for >>>>>>> what >>>>>>>>>> ends >>>>>>>>>>> up being the official Docker container I'm fine with >> either >>>>>> choice >>>>>>>>> Debian >>>>>>>>>>> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang >>> wrote: > >>>>>> Yeah. >>>>>>>> For >>>>>>>>>>> binary release, the license is a huge factor. > > On Sun, >>> May >>>>> 17, >>>>>>>> 2020 >>>>>>>>> at >>>>>>>>>>> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it >>>> comes >>>>>> to >>>>>>>>>> choosing >>>>>>>>>>> the official Apache Docker image for Heron we > > have to >>>> think >>>>>> of >>>>>>>> the >>>>>>>>>>> licensing issues first. Everything that is > installed > >>> >>>> into >>>>>> the >>>>>>>>>>> container at build time with the exception of the "FROM" >>> >>>> >>>>>>>> statement >>>>>>>>> in >>>>>>>>>>> the Dockerfile must be ALv2 compatible. OpenJDK is under >>> >>>>> GPLv2 >>>>>>> >>>>>>>> >>>>>>>>>> which >>>>>>>>>>> is not compatible with Apache. However we can get around >>> this >>>>> if >>>>>>> >>>>>>> we >>>>>>>>>>> >>>>>>>>>>> bring the OpenJDK into the container via a FROM >> statement. >>>> I've >>>>>>>> copied >>>>>>>>> a >>>>>>>>>>> >>>>>>>>>>>> reply from Dave in our dev list in a previous email. >> See >>>>> below. >>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ######## Start ######## > > Regarding OpenJDK and GPL2 - >>> here >>>>> is >>>>>>> what >>>>>>>>>> Roman >>>>>>>>>>> the VP, Legal wrote when > > answering Beam’s questions. >>> >>>> >>>>> >>>>>> >>>>>>>> Roman >>>>>>>>>>> Shaposhnik commented on LEGAL-503: > > >>>>>>>>>>> ---------------------------------------- > > > > Hey >>> [~altay] >>>>> if >>>>>>> you >>>>>>>>>> would >>>>>>>>>>> like to continue linking to the Docker release > > >> artifact >>>>> from >>>>>>> the >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> https://beam.apache.org > > you will have: > > 1. >>> Transition >>>>> to >>>>>>> the >>>>>>>>>>> official ASF dockerhub org: > > >>>>> https://hub.docker.com/u/apache >>>>>>> >>>>>>>> >>>>>>>> 2. >>>>>>>>>>> Start including that binary convenience artifact into >> your >>>> VOTE >>>>>>>> >>>>>>>>>> threads >>>>>>>>>>> on Beam releases > > 3. Make sure that all Cat-X licenses >>> are >>>>>> ONLY >>>>>>>>>> brought >>>>>>>>>>> into your > > container via FROM statements > > ######## >>> End >>>>>>>> ######## > >>>>>>>>>>>> >>>>>>>>>>>> So at the end of the day, we need to use a container >> that >>>> has >>>>>>> Java >>>>>>>>> >>>>>>>>>>> already > > installed in it so that when we create our >>>> official >>>>>>> image >>>>>>>>> we >>>>>>>>>>> can inherit > > that java containing image via a FROM >>>>> statement. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> - >>>>>>>>>>> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W >>> wrote: > >>>>>>> >>>>>>>>> >>>>>>>>> We >>>>>>>>>>> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at >>>> 10:41 >>>>>> AM >>>>>>>>>> Nicholas >>>>>>>>>>> Nezis < > > nicholas.ne...@gmail.com> > > > wrote: > > > >>>>> >>>>>>> >>>>>>>> Some >>>>>>>>>>> thoughts I'd like to add to the discussion: > > > > > > >>>> >>>>>> Debian >>>>>>>> and >>>>>>>>>>> Ubuntu are somewhat similar. Would it make sense to pick >>> >>>> one >>>>>> >>>>>>> >>>>>>>> >>>>>>>> of >>>>>>>>>>>> >>>>>>>>>>>>> those and pair with Centos? > > > > > > > > I've run >>> into >>>>>>> issues >>>>>>>>> with >>>>>>>>>>> Ubuntu images having DNS issues in > > Kubernetes. > > > >>> >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 > > > > >>>>> >>>>>>> >>>>>>>>> Ubuntu >>>>>>>>>>> 14.04 should be removed. It was end of life last year. >> For >>>> >>>>>> those >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> of >>>>>>>>>>>>>>> you using Heron in production, is anyone using >>> Ubuntu >>>>> 16? >>>>>>> My >>>>>>>>>> vote > >>>>>>>>>>> would > > > be > > > > to remove it also. For what it's >>>> worth, >>>>> my >>>>>>> use >>>>>>>>>> case >>>>>>>>>>> is in K8s and we > > use > > > > the Centos7 image on a >>>> Centos >>>>> 7 >>>>>>> OS. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>> We should do a test of the official image and >>> helm >>>>>> chart >>>>>>>> in > >>>>>>>>>>> Kubernetes > > > > (maybe with Kind >>>> https://kind.sigs.k8s.io/ >>>>> ) >>>>>> as >>>>>>>>> part >>>>>>>>>>> of our release > > > > process. This might be nice to add >>> as >>>> an >>>>>>>>>> integration >>>>>>>>>>> test. > > > > > > > > > > > > > > > > > > > > > > > > > > >>>>> >>>>>>> >>>>>>>>> >>>>>>>>> On >>>>>>>>>>> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > >>> >>>> >>>>> >>>>>> >>>>>>> That >>>>>>>>> is >>>>>>>>>> a >>>>>>>>>>> good point. We need to adjust the OS version accordingly >>> >>>> >>>>>> too. > >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sun, May 17, 2020 at 2:28 AM Windham Wong < >>> >>>> >>>>>>>>>>> windham.w...@stormeye.io > > > > > > > > > wrote: > > > >>> >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>> debian 10 has no python 2.7 support. Ubuntu >> 16.04 >>>> has >>>>>> no >>>>>>>>>> python > >>>>>>>>>>>> 3.7+ > > > > dev > > > > > > library support. hope this >>>> helps >>>>>>>> >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> -- > > > > > > Sent from myMail for Android Sunday, 17 >> May >>>>> 2020, >>>>>>>>> 11:34AM >>>>>>>>>>> +08:00 > > from > > > > > Ning > > > > > > Wang >>>>>>> wangnin...@gmail.com >>>>>>>>> : > >>>>>>>>>>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > > >We >>> have >>>>>>> talked >>>>>>>>>> about >>>>>>>>>>> it a few times in different places. Let's > > make > > > >> a >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>>>> decision here. We don't have enough resources to support >>>> many >>>>>> >>>>>>> >>>>>>>>>> docker > >>>>>>>>>>>>>>> OSes > > > > > > >(technically we only support >>>> Debian, >>>>>>>> Ubuntu, >>>>>>>>>> and >>>>>>>>>>> CentOS but each > > of > > > > them > > > > > > >has >>> multiple >>>>>>>>> versions). >>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Things we have agreed on (my >>>>>>> understanding) >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>> - for binary release, we are going to choose one >> OS >>>> one >>>>>> >>>>>>>>>> version. I >>>>>>>>>>>>>>>>>> remember that Debian 9 was the decision. An >>> open >>>>>>>> question >>>>>>>>>> is: >>>>>>>>>>> do > > we > > > > want > > > > > > to > > > > > > >switch >>> to >>>>>> Debian >>>>>>>> 10 >>>>>>>>>> now? >>>>>>>>>>> We haven't had a binary release yet, so > > > there > > > >>>>> >>>>> is >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>> still time. As the main target, I think 1. the image >>>> should >>>>>> >>>>>>> >>>>>>>>>> compile >>>>>>>>>>>>>>>>>> successfully. and 2 the image should be >>> tested: >>>>> the >>>>>>>> tools >>>>>>>>>> and >>>>>>>>>>> an > > > > example > > > > > > >topology. > > > > > > > >>>> >>>>>> >>>>>>>> >>>>>>>>> - >>>>>>>>>> for >>>>>>>>>>> source release, we haven't talked about it very much. > > >>>>> >>>>>>>>> Personally >>>>>>>>>> I >>>>>>>>>>>>>>>>>> feel it is a reasonable expectation that, if >>> we >>>>> have >>>>>>> the >>>>>>>>>>> Docker > > file > > > > in > > > > > > the > > > > > > >>>> core >>>>>>> folders, >>>>>>>>> it >>>>>>>>>>> should at least compile. Otherwise, it's a > failed > > >>>> >>>>>> build > >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> hence a failed release. We are not going to test run >> the >>>>>> tools > >>>>>>>> and >>>>>>>>>>>> >>>>>>>>>>>>>>> examples > > > > > > >for each release though. > >>> >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>> Finally, a question about OS versions. I believe >> at >>>>> least >>>>>>> two >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> releases > > > > > > >should be supported because most >>> people >>>>>> don't >>>>>>>>>> upgrade >>>>>>>>>>> to the > > latest > > > > > > version > > > > > > >when >> it >>> is >>>>> out >>>>>>> in >>>>>>>>> more >>>>>>>>>>> serious scenarios, hence IMO the second > > > oldest > > >>> >>>> >>>>> >>>>>> >>>>>>>> could >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>> often be more useful than the latest one from a >>>>>>> convenience >>>>>>>>> >>>>>>>>>>> point > > of > > > > > view. > > > > > > >On the other >>> hand, >>>>> this >>>>>>> is >>>>>>>>> too >>>>>>>>>>> flexible, and we will spend time > > > > thinking > > > > >>>>> >>>>>>>> about >>>>>>>> it >>>>>>>>>>> again. I think a clear guideline could be helpful to > >> us. >>>> >>>>> >>>>>> >>>>>>> For >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>> example: > > > > > > >- We only include only >> Debian >>>>>> (popular >>>>>>>> in >>>>>>>>>> the >>>>>>>>>>> server world) and > > > Ubuntu > > > > > > >(popular in >>>> servers >>>>>> and >>>>>>>>>>> workstations). > > > > > > >- for the main OS (Debian) we >>>>> choose >>>>>>> the >>>>>>>>>> latest >>>>>>>>>>> to compile and > > test > > > > for > > > > > > our > > >>>> >>>>>> >>>>>>>>> binary >>>>>>>>>>> release. > > > > > > >- for a given OS, only two versions >>>> (most >>>>>>>> likely >>>>>>>>>> the >>>>>>>>>>> most and > the > > > > second > > > > > > >recent >> releases) >>>> are >>>>>>>>> included >>>>>>>>>> in >>>>>>>>>>> the core source code and they > > need > > > to > > > > > >>> be >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> fixed ASAP if they don't compile. > > > > > > >- OSs and >>>> other >>>>>>>>> versions >>>>>>>>>>> could be included in a special folder > and > > > but > > >>>>> >>>>>>> >>>>>>>>>> there >>>>>>>>>> is >>>>>>>>>>> no guarantee that the code will compile. Fixes from the > >>>>> >>>>>>> >>>>>>>>>> community >>>>>>>>>>>>>>>>>> are welcome. > > > > > > > > > > > > > > > > >>>>> >>>>>>> >>>>>>>>> How >>>>>>>>> do >>>>>>>>>>> you guys think? > > > > > > >--ning > > > > > > > > > > >>>> >>>>>> >>>>>>>> >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>
