On Thu, Jul 25, 2019 at 5:01 PM Michael Gugino <mgug...@redhat.com> wrote:

> I don't really view the 'bucket of parts' and 'complete solution' as
> competing ideas.  It would be nice to build the 'complete solution'
> from the 'bucket of parts' in a reproducible, customizable manner.
> "How is this put together" should be easily followed, enough so that
> someone can 'put it together' on their own infrastructure without
> having to be an expert in designing and configuring the build system.
>
> IMO, if I can't build it, I don't own it.  In 3.x, I could compile all
> the openshift-specific bits from source, I could point at any
> repository I wanted, I could point to any image registry I wanted, I
> could use any distro I wanted.  I could replace the parts I wanted to;
> or I could just run it as-is from the published sources and not worry
> about replacing things.  I even built Fedora Atomic host rpm-trees
> with all the kublet bits pre-installed, similar to what we're doing
> with CoreOS now in 3.x.  It was a great experience, building my own
> system images and running updates was trivial.
>
> I wish we weren't EOL'ing the Atomic Host in Fedora.  It offered a lot
> of flexibility and easy to use tooling.
>
> So maybe what we are asking here is:

   - opinionated OCP 4 philosophy => OKD 4 + FCOS (IPI and UPI) using
   ignition, CVO etc
   - DYI kube philosophy reusing as many v4 components but with your own
   preferred operating system


In terms of approach, priority i think is fair to adopt a baby steps
approach where:

   - phase 1 = try to get out OKD 4 + FCOS asap so folks can start build up
   the knowledge around operating the new solution in a full production env
   - phase 2 = once experience/ knowledge was built up then we can crack on
   with reverse eng and see what we can swap etc.





> On Thu, Jul 25, 2019 at 9:51 AM Clayton Coleman <ccole...@redhat.com>
> wrote:
> >
> > > On Jul 25, 2019, at 4:19 AM, Aleksandar Lazic <
> openshift-li...@me2digital.com> wrote:
> > >
> > > HI.
> > >
> > >> Am 25.07.2019 um 06:52 schrieb Michael Gugino:
> > >> I think FCoS could be a mutable detail.  To start with, support for
> > >> plain-old-fedora would be helpful to make the platform more portable,
> > >> particularly the MCO and machine-api.  If I had to state a goal, it
> > >> would be "Bring OKD to the largest possible range of linux distros to
> > >> become the defacto implementation of kubernetes."
> > >
> > > I agree here with Michael. As FCoS or in general CoS looks technical a
> good idea
> > > but it limits the flexibility of possible solutions.
> > >
> > > For example when you need to change some system settings then you will
> need to
> > > create a new OS Image, this is not very usable in some environments.
> >
> > I think something we haven’t emphasized enough is that openshift 4 is
> > very heavily structured around changing the cost and mental model
> > around this.  The goal was and is to make these sorts of things
> > unnecessary.  Changing machine settings by building golden images is
> > already the “wrong” (expensive and error prone) pattern - instead, it
> > should be easy to reconfigure machines or to launch new containers to
> > run software on those machines.  There may be two factors here at
> > work:
> >
> > 1. Openshift 4 isn’t flexible in the ways people want (Ie you want to
> > add an rpm to the OS to get a kernel module, or you want to ship a
> > complex set of config and managing things with mcd looks too hard)
> > 2. You want to build and maintain these things yourself, so the “just
> > works” mindset doesn’t appeal.
> >
> > The initial doc alluded to the DIY / bucket of parts use case (I can
> > assemble this on my own but slightly differently) - maybe we can go
> > further now and describe the goal / use case as:
> >
> > I want to be able to compose my own Kubernetes distribution, and I’m
> > willing to give up continuous automatic updates to gain flexibility in
> > picking my own software
> >
> > Does that sound like it captures your request?
> >
> > Note that a key reason why the OS is integrated is so that we can keep
> > machines up to date and do rolling control plane upgrades with no
> > risk.  If you take the OS out of the equation the risk goes up
> > substantially, but if you’re willing to give that up then yes, you
> > could build an OKD that doesn’t tie to the OS.  This trade off is an
> > important one for folks to discuss.  I’d been assuming that people
> > *want* the automatic and safe upgrades, but maybe that’s a bad
> > assumption.
> >
> > What would you be willing to give up?
> >
> > >
> > > It would be nice to have the good old option to use the ansible
> installer to
> > > install OKD/Openshift on other Linux distribution where ansible is
> able to run.
> > >
> > >> Also, it would be helpful (as previously stated) to build communities
> > >> around some of our components that might not have a place in the
> > >> official kubernetes, but are valuable downstream components
> > >> nevertheless.
> > >>
> > >> Anyway, I'm just throwing some ideas out there, I wouldn't consider my
> > >> statements as advocating strongly in any direction.  Surely FCoS is
> > >> the natural fit, but I think considering other distros merits
> > >> discussion.
> > >
> > > +1
> > >
> > > Regards
> > > Aleks
> > >
> > >
> > >>> On Wed, Jul 24, 2019 at 9:23 PM Clayton Coleman <ccole...@redhat.com>
> wrote:
> > >>>
> > >>>> On Jul 24, 2019, at 9:14 PM, Michael Gugino <mgug...@redhat.com>
> wrote:
> > >>>>
> > >>>> I think what I'm looking for is more 'modular' rather than DIY.  CVO
> > >>>> would need to be adapted to separate container payload from host
> > >>>> software (or use something else), and maintaining cross-distro
> > >>>> machine-configs might prove tedious, but for the most part, rest of
> > >>>> everything from the k8s bins up, should be more or less the same.
> > >>>>
> > >>>> MCD is good software, but there's not really much going on there
> that
> > >>>> can't be ported to any other OS.  MCD downloads a payload, extracts
> > >>>> files, rebases ostree, reboots host.  You can do all of those steps
> > >>>> except 'rebases ostree' on any distro.  And instead of 'rebases
> > >>>> ostree', we could pull down a container that acts as a local repo
> that
> > >>>> contains all the bits you need to upgrade your host across releases.
> > >>>> Users could do things to break this workflow, but it should
> otherwise
> > >>>> work if they aren't fiddling with the hosts.  The MCD payload
> happens
> > >>>> to embed an ignition payload, but it doesn't actually run ignition,
> > >>>> just utilizes the file format.
> > >>>>
> > >>>> From my viewpoint, there's nothing particularly special about
> ignition
> > >>>> in our current process either.  I had the entire OCP 4 stack running
> > >>>> on RHEL using the same exact ignition payload, a minimal amount of
> > >>>> ansible (which could have been replaced by cloud-init userdata),
> and a
> > >>>> small python library to parse the ignition files.  I was also
> building
> > >>>> repo containers for 3.10 and 3.11 for Fedora.  Not to say the
> > >>>> OpenShift 4 experience isn't great, because it is.  RHEL CoreOS +
> OCP
> > >>>> 4 came together quite nicely.
> > >>>>
> > >>>> I'm all for 'not managing machines' but I'm not sure it has to look
> > >>>> exactly like OCP.  Seems the OCP installer and CVO could be
> > >>>> adapted/replaced with something else, MCD adapted, pretty much
> > >>>> everything else works the same.
> > >>>
> > >>> Sure - why?  As in, what do you want to do?  What distro do you want
> > >>> to use instead of fcos?  What goals / outcomes do you want out of the
> > >>> ability to do whatever?  Ie the previous suggestion (the auto
> updating
> > >>> kube distro) has the concrete goal of “don’t worry about security /
> > >>> updates / nodes and still be able to run containers”, and fcos is a
> > >>> detail, even if it’s an important one.  How would you pitch the
> > >>> alternative?
> > >>>
> > >>>
> > >>>>
> > >>>>> On Wed, Jul 24, 2019 at 12:05 PM Clayton Coleman <
> ccole...@redhat.com> wrote:
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> On Wed, Jul 24, 2019 at 10:40 AM Michael Gugino <
> mgug...@redhat.com> wrote:
> > >>>>>>
> > >>>>>> I tried FCoS prior to the release by using the assembler on
> github.
> > >>>>>> Too much secret sauce in how to actually construct an image.  I
> > >>>>>> thought atomic was much more polished, not really sure what the
> > >>>>>> value-add of ignition is in this usecase.  Just give me a way to
> build
> > >>>>>> simple image pipelines and I don't need ignition.  To that end,
> there
> > >>>>>> should be an okd 'spin' of FCoS IMO.  Atomic was dead simple to
> build
> > >>>>>> ostree repos for.  I'd prefer to have ignition be opt-in.  Since
> we're
> > >>>>>> supporting the mcd-once-from to parse ignition on RHEL, we don't
> need
> > >>>>>> ignition to actually install okd.  To me, it seems FCoS was
> created
> > >>>>>> just to have a more open version of RHEL CoreOS, and I'm not sure
> FCoS
> > >>>>>> actually solves anyone's needs relative to atomic.  It feels like
> we
> > >>>>>> jumped the shark on this one.
> > >>>>>
> > >>>>>
> > >>>>> That’s feedback that’s probably something you should share in the
> fcos forums as well.  I will say that I find the OCP + RHEL experience
> unsatisfying and doesn't truly live up to what RHCOS+OCP can do (since it
> lacks the key features like ignition and immutable hosts).  Are you saying
> you'd prefer to have more of a "DIY kube bistro" than the "highly
> opinionated, totally integrated OKD" proposal?  I think that's a good
> question the community should get a chance to weigh in on (in my original
> email that was the implicit question - do you want something that looks
> like OCP4, or something that is completely different).
> > >>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> I'd like to see OKD be distro-independent.  Obviously Fedora
> should be
> > >>>>>> our primary target (I'd argue Fedora over FCoS), but I think it
> should
> > >>>>>> be true upstream software in the sense that apache2 http server is
> > >>>>>> upstream and not distro specific.  To this end, perhaps it makes
> sense
> > >>>>>> to consume k/k instead of openshift/origin for okd.  OKD should be
> > >>>>>> free to do wild and crazy things independently of the enterprise
> > >>>>>> product.  Perhaps there's a usecase for treating k/k vs
> > >>>>>> openshift/origin as a swappable base layer.
> > >>>>>
> > >>>>>
> > >>>>> That’s even more dramatic a change from OKD even as it was in
> 3.x.  I’d be happy to see people excited about reusing cvo / mcd and be
> able to mix and match, but most of the things here would be a huge
> investment to build.  In my original email I might call this the “I want to
> build my own distro" - if that's what people want to build, I think we can
> do things to enable it.  But it would probably not be "openshift" in the
> same way.
> > >>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> It would be nice to have a more native kubernetes place to
> develop our
> > >>>>>> components against so we can upstream them, or otherwise just
> build a
> > >>>>>> solid community around how we think kubernetes should be deployed
> and
> > >>>>>> consumed.  Similar to how Fedora has a package repository, we
> should
> > >>>>>> have a Kubernetes component repository (I realize operatorhub
> fulfulls
> > >>>>>> some of this, but I'm talking about a place for OLM and things
> like
> > >>>>>> MCD to live).
> > >>>>>
> > >>>>>
> > >>>>> MCD is really tied to the OS.  The idea of a generic MCD seems
> like it loses the value of MCD being specific to an OS.
> > >>>>>
> > >>>>> I do think there are two types of components we have - things
> designed to work well with kube, and things designed to work well with
> "openshift the distro".  The former can be developed against Kube (like
> operator hub / olm) but the latter doesn't really make sense to develop
> against unless it matches what is being built.  In that vein, OKD4 looking
> not like OCP4 wouldn't benefit you or the components.
> > >>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> I think we could integrate with existing package managers via a
> > >>>>>> 'repo-in-a-container' type strategy for those not using ostree.
> > >>>>>
> > >>>>>
> > >>>>> A big part of openshift 4 is "we're tired of managing machines".
> It sounds like you are arguing for "let people do whatever", which is
> definitely valid (but doesn't sound like openshift 4).
> > >>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> As far as slack vs IRC, I vote IRC or any free software solution
> (but
> > >>>>>> my preference is IRC because it's simple and I like it).
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Michael Gugino
> > >>>> Senior Software Engineer - OpenShift
> > >>>> mgug...@redhat.com
> > >>>> 540-846-0304
> > >>
> > >>
> > >>
> > >
>
>
>
> --
> Michael Gugino
> Senior Software Engineer - OpenShift
> mgug...@redhat.com
> 540-846-0304
>
> _______________________________________________
> dev mailing list
> dev@lists.openshift.redhat.com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>
_______________________________________________
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Reply via email to