Hi All,
Based on what I've read on this thread a combination of CentOS/Debian would
be the better options for us to support multiple versions for the
community. We will also support one version of Ubuntu.
So..... I will start a vote in 72 hours that calls on the vote on this.
I'm going to write down my understanding of what this thread covers to get
some actionable topics to vote on. If I've written anything down
incorrectly please speak up before we start a vote.
Debian:
- Debian10 is the official docker image to support and be published to
Docker Hub, we will also support Debian9.
CentOS:
- We will support CentOS7. Note, I do not think it has native
Python3.7 support, but we are also not there yet.
Ubuntu:
- We will still support one version. At this time the version is 18.04.
We will upgrade supported versions of each OS as updates/deprecation of
support are needed.
Again let's wait 72 hours before starting a vote. Also please check that
the statements I wrote down are what you understood!
See below for notes on lazy consensus:
https://community.apache.org/committers/lazyConsensus.html
- Josh
On Wed, May 20, 2020 at 9:41 PM Ning Wang <[email protected]> wrote:
> +1
>
> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <[email protected]>
> wrote:
>
> > I like those options.
> >
> > On Wed, May 20, 2020 at 10:15 PM Josh Fischer <[email protected]>
> wrote:
> >
> > > Let's wait 3 more days for anyone to give input. Then we can start a
> > vote
> > > to make a final decision on which containers for us to support. Keep
> in
> > > mind that even though we do need to plan for Python3 support (There is
> > work
> > > in progress now) we do not need it as of yet.
> > >
> > > - Josh
> > >
> > > On Wed, May 20, 2020 at 8:34 PM thinker0 <[email protected]> wrote:
> > >
> > > > My experience is mainly for CentOS 7, so I think it would be nice if
> it
> > > was
> > > > included.
> > > >
> > > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <[email protected]>님이 작성:
> > > >
> > > > > Well thought out Windham. I like how you narrowed the criteria
> down
> > to
> > > > > Debian10/CentOS7(depending on PY3 support) thinking about overall
> > > support
> > > > > and what others would need/want.
> > > > >
> > > > > +1
> > > > >
> > > > > - Josh
> > > > >
> > > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > [email protected]
> > > >
> > > > > wrote:
> > > > >
> > > > > > My experience is that, people tends to pick their container image
> > OS
> > > > > based
> > > > > > on few criterias only:
> > > > > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > > > > ubuntu/centos. alpine is always the smallest but not much library
> > > > > provided
> > > > > > and require to compile manually. debian is always good because it
> > has
> > > > > > apt-get to provide loads of library. centos is cool as debian
> > because
> > > > it
> > > > > > also has good yum repo. ubuntu is always the last because it is
> > fat,
> > > > but
> > > > > it
> > > > > > provides very large range of libraries to use.
> > > > > > 2) kernel feature, as mentioned last email, ubuntu has issue with
> > DNS
> > > > and
> > > > > > some sort of network discover issues. I remember alpine has some
> as
> > > > well,
> > > > > > because it is very plain, requires many fine tunes be done
> > manually.
> > > in
> > > > > > this case, centos and debian should win.
> > > > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3
> > > > > different
> > > > > > streams that use different native tools. Poeple just pick their
> > base
> > > > > image
> > > > > > based on what they often use in their life. However, I see a lot
> > > debian
> > > > > and
> > > > > > alpine on docker hub because they are just smaller in size.
> > > > > > 4. library support. Python library is a very good example in this
> > > case.
> > > > > As
> > > > > > Heron is turning to Python 3, which specifically Python 3.7+,
> some
> > > > > > discontinued based OS version does not provide the support
> anymore.
> > > We
> > > > > have
> > > > > > tested out that, debian9 supports only up to Python3.5, ubuntu
> > 14.04
> > > > does
> > > > > > not support python3 and ubuntu 16.04 supports up to Python 3.5
> > only.
> > > > > >
> > > > > > I am not a big fan of using centos for docker. Concluding all 4
> > > points
> > > > > > above, I believe keeping centos7 (if it supports python37),
> > debian10
> > > > (not
> > > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the best
> > > solution
> > > > > at
> > > > > > the moment. We have to cut off the old images due to our library
> > > > upgrade
> > > > > > and this is unavoidable. Otherwise, we would have tons of issues
> > > > sending
> > > > > in
> > > > > > about library support in the future and the workload could be a
> lot
> > > > more
> > > > > > for just answering questions.
> > > > > >
> > > > > > Windham Wong
> > > > > > Co-Founder, Technical Director of
> > > > > > Stormeye.io, Hong Kong Managed Security Operation Center Limited
> > > > > > Email // [email protected] (mailto:
> [email protected]
> > )
> > > > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > > >
> > > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <[email protected]>
> > wrote:
> > > > > > > Hi All, I think using CentOS and either Debian or Ubuntu is a
> > good
> > > > > idea.
> > > > > > I would pick Debian over Ubuntu for the reason that Nick pointed
> > out
> > > > in a
> > > > > > previous email of DNS issues in Kubernetes. I"ve copy and pasted
> a
> > > > > section
> > > > > > of Nick's previous email. ### Start I've run into issues with
> > Ubuntu
> > > > > images
> > > > > > having DNS issues in Kubernetes.
> > > > > > https://github.com/kubernetes/kubeadm/issues/787 ### End As for
> > what
> > > > > ends
> > > > > > up being the official Docker container I'm fine with either
> choice
> > > > Debian
> > > > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: >
> Yeah.
> > > For
> > > > > > binary release, the license is a huge factor. > > On Sun, May 17,
> > > 2020
> > > > at
> > > > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes
> to
> > > > > choosing
> > > > > > the official Apache Docker image for Heron we > > have to think
> of
> > > the
> > > > > > licensing issues first. Everything that is > installed > > into
> the
> > > > > > container at build time with the exception of the "FROM" > >
> > > statement
> > > > in
> > > > > > the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2
> >
> > >
> > > > > which
> > > > > > is not compatible with Apache. However we can get around this if
> >
> > we
> > > > > >
> > > > > > bring the OpenJDK into the container via a FROM statement. I've
> > > copied
> > > > a
> > > > > >
> > > > > > > reply from Dave in our dev list in a previous email. See below.
> > > >
> > > > > >
> > > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here is
> > what
> > > > > Roman
> > > > > > the VP, Legal wrote when > > answering Beam’s questions. > > > >
> > > Roman
> > > > > > Shaposhnik commented on LEGAL-503: > >
> > > > > > ---------------------------------------- > > > > Hey [~altay] if
> > you
> > > > > would
> > > > > > like to continue linking to the Docker release > > artifact from
> > the
> > > >
> > > > >
> > > > > > https://beam.apache.org > > you will have: > > 1. Transition to
> > the
> > > > > > official ASF dockerhub org: > > https://hub.docker.com/u/apache
> >
> > >
> > > 2.
> > > > > > Start including that binary convenience artifact into your VOTE
> > >
> > > > > threads
> > > > > > on Beam releases > > 3. Make sure that all Cat-X licenses are
> ONLY
> > > > > brought
> > > > > > into your > > container via FROM statements > > ######## End
> > > ######## >
> > > > > > >
> > > > > > > So at the end of the day, we need to use a container that has
> > Java
> > > >
> > > > > > already > > installed in it so that when we create our official
> > image
> > > > we
> > > > > > can inherit > > that java containing image via a FROM statement.
> >
> > >
> > > >
> > > > >
> > > > > -
> > > > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > >
> > > >
> > > > We
> > > > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41
> AM
> > > > > Nicholas
> > > > > > Nezis < > > [email protected]> > > > wrote: > > > > > > >
> > > Some
> > > > > > thoughts I'd like to add to the discussion: > > > > > > > >
> Debian
> > > and
> > > > > > Ubuntu are somewhat similar. Would it make sense to pick > one >
> >
> > >
> > > of
> > > > > > >
> > > > > > > > those and pair with Centos? > > > > > > > > I've run into
> > issues
> > > > with
> > > > > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > >
> > > > Ubuntu
> > > > > > 14.04 should be removed. It was end of life last year. For >
> those
> > >
> > > >
> > > > >
> > > > > of
> > > > > > > > > > you using Heron in production, is anyone using Ubuntu 16?
> > My
> > > > > vote >
> > > > > > would > > > be > > > > to remove it also. For what it's worth, my
> > use
> > > > > case
> > > > > > is in K8s and we > > use > > > > the Centos7 image on a Centos 7
> > OS.
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > We should do a test of the official image and helm
> chart
> > > in >
> > > > > > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ )
> as
> > > > part
> > > > > > of our release > > > > process. This might be nice to add as an
> > > > > integration
> > > > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
> > > >
> > > > On
> > > > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > >
> > That
> > > > is
> > > > > a
> > > > > > good point. We need to adjust the OS version accordingly > >
> too. >
> > > > >
> > > > > > >
> > > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > > > > > [email protected] > > > > > > > > > wrote: > > > > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has
> no
> > > > > python >
> > > > > > > 3.7+ > > > > dev > > > > > > library support. hope this helps
> > >
> > > > >
> > > > > > >
> > > > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020,
> > > > 11:34AM
> > > > > > +08:00 > > from > > > > > Ning > > > > > > Wang
> > [email protected]
> > > > : >
> > > > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have
> > talked
> > > > > about
> > > > > > it a few times in different places. Let's > > make > > > a > > >
> >
> > >
> > > >
> > > > > > >decision here. We don't have enough resources to support many >
> >
> > > > > docker >
> > > > > > > > > > OSes > > > > > > >(technically we only support Debian,
> > > Ubuntu,
> > > > > and
> > > > > > CentOS but each > > of > > > > them > > > > > > >has multiple
> > > > versions).
> > > > > >
> > > > > > > > > > > > > > > > > > >Things we have agreed on (my
> > understanding)
> > > >
> > > > >
> > > > > >
> > > > > > > > > >- for binary release, we are going to choose one OS one >
> > > > > version. I
> > > > > > > > > > > > >remember that Debian 9 was the decision. An open
> > > question
> > > > > is:
> > > > > > do > > we > > > > want > > > > > > to > > > > > > >switch to
> Debian
> > > 10
> > > > > now?
> > > > > > We haven't had a binary release yet, so > > > there > > > > > is
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >still time. As the main target, I think 1. the image should >
> >
> > > > > compile
> > > > > > > > > > > > >successfully. and 2 the image should be tested: the
> > > tools
> > > > > and
> > > > > > an > > > > example > > > > > > >topology. > > > > > > > > > > >
> > >
> > > >-
> > > > > for
> > > > > > source release, we haven't talked about it very much. > > > >
> > > > Personally
> > > > > I
> > > > > > > > > > > > >feel it is a reasonable expectation that, if we have
> > the
> > > > > > Docker > > file > > > > in > > > > > > the > > > > > > >core
> > folders,
> > > > it
> > > > > > should at least compile. Otherwise, it's a > failed > > > >
> build >
> > > > >
> > > > > > >
> > > > > > > >hence a failed release. We are not going to test run the
> tools >
> > > and
> > > > > > >
> > > > > > > > > > examples > > > > > > >for each release though. > > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >Finally, a question about OS versions. I believe at least
> > two
> > > >
> > > > >
> > > > > >
> > > > > > releases > > > > > > >should be supported because most people
> don't
> > > > > upgrade
> > > > > > to the > > latest > > > > > > version > > > > > > >when it is out
> > in
> > > > more
> > > > > > serious scenarios, hence IMO the second > > > oldest > > > > > >
> > > could
> > > > >
> > > > > >
> > > > > > > > > > >often be more useful than the latest one from a
> > convenience
> > > >
> > > > > > point > > of > > > > > view. > > > > > > >On the other hand, this
> > is
> > > > too
> > > > > > flexible, and we will spend time > > > > thinking > > > > > >
> > >about
> > > it
> > > > > > again. I think a clear guideline could be helpful to > us. > > >
> > For
> > > >
> > > > >
> > > > > >
> > > > > > > > > >example: > > > > > > >- We only include only Debian
> (popular
> > > in
> > > > > the
> > > > > > server world) and > > > Ubuntu > > > > > > >(popular in servers
> and
> > > > > > workstations). > > > > > > >- for the main OS (Debian) we choose
> > the
> > > > > latest
> > > > > > to compile and > > test > > > > for > > > > > > our > > > > > >
> > > >binary
> > > > > > release. > > > > > > >- for a given OS, only two versions (most
> > > likely
> > > > > the
> > > > > > most and > the > > > > second > > > > > > >recent releases) are
> > > > included
> > > > > in
> > > > > > the core source code and they > > need > > > to > > > > > be > >
> >
> > >
> > > >
> > > > >
> > > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and other
> > > > versions
> > > > > > could be included in a special folder > and > > > but > > > > > >
> > > > >there
> > > > > is
> > > > > > no guarantee that the code will compile. Fixes from the > > > > >
> > > > > community
> > > > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > >
> > > >How
> > > > do
> > > > > > you guys think? > > > > > > >--ning > > > > > > > > > > > > > >
> > >
> > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
