Re: [atomic-devel] Atomic as Base OS for Standalone Appliance

[Dusty Mabe]

On 05/30/2018 09:52 AM, Scott McCarty wrote:
> 

> My apologies, I just had this sitting in my inbox forever and didn't see 
> a response. I get that subconscious stressful feeling when I know there 
> are unanswered questions on a mailing anywhere the Internet ;-)
>>

I get the same feeling. Now I know I can send the links to you and clear
my conscious. :) 

Dusty 

Re: [atomic-devel] Atomic as Base OS for Standalone Appliance

[Scott McCarty]

On 05/26/2018 06:50 PM, Dusty Mabe wrote:


On 05/25/2018 04:34 PM, Scott McCarty wrote:

Shane,

      I see nobody has responded. To be honest, I read your email three
times, and I can't quite figure out your use case. I am guessing others
may be heads down working on the new Red Hat CoreOS and hence haven't
had time to dig into this.

*cough*
https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2018-May/msg00038.html
*cough*
My apologies, I just had this sitting in my inbox forever and didn't see 
a response. I get that subconscious stressful feeling when I know there 
are unanswered questions on a mailing anywhere the Internet ;-)


:)


I will be down in Raleigh on the afternoon/evening of June 5th and would
love to meet up and chat. Happy to meet up late afternoon, or early
evening. If you are around feel free to respond to me off list.

Let me know if you guys schedule something.


--

Scott McCarty, RHCA

Technical Product Marketing: Containers

Email: smcca...@redhat.com

Phone: 312-660-3535

Cell: 330-807-1043

Web: http://crunchtools.com

When should you split your application into multiple containers? 
http://red.ht/22xKw9i

Re: [atomic-devel] Atomic as Base OS for Standalone Appliance

[Dusty Mabe]

On 05/25/2018 04:34 PM, Scott McCarty wrote:
> Shane,
> 
>      I see nobody has responded. To be honest, I read your email three 
> times, and I can't quite figure out your use case. I am guessing others 
> may be heads down working on the new Red Hat CoreOS and hence haven't 
> had time to dig into this.

*cough*
https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2018-May/msg00038.html
*cough*

:)

> I will be down in Raleigh on the afternoon/evening of June 5th and would 
> love to meet up and chat. Happy to meet up late afternoon, or early 
> evening. If you are around feel free to respond to me off list.

Let me know if you guys schedule something.

Re: [atomic-devel] Atomic as Base OS for Standalone Appliance

[Shane O'Donnell]

Inline...

On 5/22/18, 12:30 PM, "atomic-devel-boun...@projectatomic.io on behalf of 
Colin Walters"  wrote:

Hi Shane,

On Mon, May 21, 2018, at 3:09 PM, Shane O'Donnell wrote:
> Hey All –
>
> We’re building an IoT edge device based on CentOS Atomic Host

Cool; without knowing more it feels like this falls in between "server"
and "device"?   If it's more "device" like then the newly formed Fedora
IoT group is intended to be the place, although as of yet as far as I
can tell there isn't even a mailing list?

I'm tracking that project now, too, but this is shipping on a quad-core x86_64 
w/ 4GB RAM & 256GB SSD.  Definitely more "server" more than "device", but 
anemic when compared to real servers.

> As an example, we’re trying to follow the “transactional” nature of
> Atomic, but we’re finding there’s no real “commit”.  When we need to
> change something in the OS itself,

A big topic here is whether you derive from the CAH ostree commits 
(hostimages),
or whether you do a custom compose/build.  The latter is fairly 
straightforward
and gives you full control.

We're doing a custom compose, but our configs still track pretty closely with 
CAH.

> we end up stuck with a “hotfix”
> flagged deployment that we can’t really “commit” to clear the flag,
> which leaves us unable to make follow-on transactional changes.

It depends on what you want to do - we could probably fairly easily
support a model where there's one or more persistent "layer"s.  I just
filed this: 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fprojectatomic%2Frpm-ostree%2Fissues%2F1370=01%7C01%7Csodonnell%40cree.com%7C8a47a4ebfe5741ab723808d5c0014905%7Ca326555ee95e4f1b87f747eba52c598e%7C0=78svYf5jTFPuft1%2BjZnp1%2FzZGQ0OebmtMWcDgq5RsjQ%3D=0

But again it depends on what content you want there.

We've run into issues in a couple of different scenarios, but as a 
representative use case:

Most of our containers have bind-mounted volumes.  We use Ansible to create 
those directories in advance, and a number of them get certs created/added to 
the directories.  We were originally doing this early in our Ansible 
processing, then after manipulating kernel args with "ex kargs", found a number 
of our changes to /etc were lost after a reboot.  We re-ordered things to get 
around this and introduced a new reboot to effectively 'commit' our changes, 
but we were hoping to find some way to allow manipulation of the current 
deployment with a with a "commit" step at the end.

Allowing a "persist overlay" sounds like it would allow us to force the "merge" 
on demand -- which would certainly address this situation.

> We are also in the process of migrating our Ansible-based platform
> customization and software-load (i.e., Docker containers managed via
> Docker Compose) which seems somewhat unnatural in the Atomic world.

Hmm.  Today we do support Ansible although most of our efforts
as you noticed are oriented towards Kubernetes.

I've found a few sparse docs on the support for Ansible as well as watched a 
YouTube video on it, but I may have overlooked something.  I was also imagining 
a scenario where our Ansible tasks/playbooks could be modal, where we run a 
task (e.g., ostree admin unlock), continue running tasks that would manipulate 
the current deployment (or overlay), then run a task (e.g., ostree admin lock) 
to commit the current deployment (or merge the current overlay).  Entering with 
this wrong-headed mindset may have stymied us a bit as well.

> The docs seem pretty clearly focused on using Atomic as the OS for a
> cluster, but our standalone deployments seem to leave us with a lot of
> questions.  I think the following list is a reasonable summary of the
> big questions we haven’t really figured out yet:
>
>
>   *   Is there a “commit” that would allow us to commit the current
> “hotfix” to the current deployment?

Not today, though the above issue covers some of it.   On IRC you mentioned
kernel arguments; we could add `--inplace` or something.  That'd likely
be fairly easy to do.  Kernel argument management is a gap for sure.

`--inplace` would work.

On a somewhat related note, we found that building a new GRUB config was broken 
and was creating a grub.cfg that tried to load "linux16" and "initrd16" vs. 
"linuxefi"/"initrdefi", but I think this may be a broader CentOS problem and 
not specific to CAH.

>   *   All of our appliances start with a common base image.  What’s the
> recommended approach for changing kernel arguments so they appear in the
> common base image?

ostree today does not store kernel arguments in the commit.  See also

Re: [atomic-devel] Atomic as Base OS for Standalone Appliance

[Dusty Mabe]

On 05/21/2018 03:09 PM, Shane O'Donnell wrote:
> Hey All –
> 

Hey Shane. I think walters answered most of your questions already but
I'm glad to see you around in the community!

> 
>  
> 
> P.S.  If you’re in the Raleigh, NC area, I’d happily buy lunch or a beer (or 
> both!) for a chance to pick someone’s brain on some of these issues.

/me raises hand! Let's get together sometime and chat. Coordinate in IRC.

Dusty 

Re: [atomic-devel] Atomic as Base OS for Standalone Appliance

[Colin Walters]

Hi Shane,

On Mon, May 21, 2018, at 3:09 PM, Shane O'Donnell wrote:
> Hey All –
> 
> We’re building an IoT edge device based on CentOS Atomic Host 

Cool; without knowing more it feels like this falls in between "server"
and "device"?   If it's more "device" like then the newly formed Fedora
IoT group is intended to be the place, although as of yet as far as I
can tell there isn't even a mailing list?

> As an example, we’re trying to follow the “transactional” nature of 
> Atomic, but we’re finding there’s no real “commit”.  When we need to 
> change something in the OS itself, 

A big topic here is whether you derive from the CAH ostree commits (hostimages),
or whether you do a custom compose/build.  The latter is fairly straightforward
and gives you full control.

> we end up stuck with a “hotfix” 
> flagged deployment that we can’t really “commit” to clear the flag, 
> which leaves us unable to make follow-on transactional changes.

It depends on what you want to do - we could probably fairly easily
support a model where there's one or more persistent "layer"s.  I just
filed this: https://github.com/projectatomic/rpm-ostree/issues/1370

But again it depends on what content you want there.

> We are also in the process of migrating our Ansible-based platform 
> customization and software-load (i.e., Docker containers managed via 
> Docker Compose) which seems somewhat unnatural in the Atomic world.

Hmm.  Today we do support Ansible although most of our efforts
as you noticed are oriented towards Kubernetes.  

> The docs seem pretty clearly focused on using Atomic as the OS for a 
> cluster, but our standalone deployments seem to leave us with a lot of 
> questions.  I think the following list is a reasonable summary of the 
> big questions we haven’t really figured out yet:
> 
> 
>   *   Is there a “commit” that would allow us to commit the current 
> “hotfix” to the current deployment?

Not today, though the above issue covers some of it.   On IRC you mentioned
kernel arguments; we could add `--inplace` or something.  That'd likely
be fairly easy to do.  Kernel argument management is a gap for sure.

>   *   All of our appliances start with a common base image.  What’s the 
> recommended approach for changing kernel arguments so they appear in the 
> common base image?

ostree today does not store kernel arguments in the commit.  See also
https://github.com/ostreedev/ostree/issues/479

The high level thing to understand is that the design mostly is that
rpm-ostree can be used in the same places/same way as one might use
"classic" package managers like yum/apt.  Since today one doesn't
store kernel arguments in rpm/deb packages, ostree also didn't do it
by default.  Instead you use kickstart or config management or whatever.
But that doesn't mean we couldn't!   Like I said in the issue, it'd
probably make sense to do, but we'd have to work through some overlap
in terms of how it works with Anaconda, etc.

See also one of our oldest issues:
https://github.com/projectatomic/rpm-ostree/issues/471

> Note that we’re both invested and committed to using Atomic, and we’d 
> like to offer up a resource to help flesh out the “standalone” side of 
> the documentation so we both have some agreed upon contract between 
> users and core devs, as well as to better enable new users with this use 
> case.

That's always nice!  

[atomic-devel] Atomic as Base OS for Standalone Appliance

[Shane O'Donnell]

Hey All –

We’re building an IoT edge device based on CentOS Atomic Host and as seasoned 
sysadmins/developers, we’re trying to break ourselves of “normal server admin” 
habits and trying to get more into the Atomic flow.  We’ve already stopped 
doing a lot of stupid things (e.g., remounting /usr rw), but some things still 
seem impossible.

As an example, we’re trying to follow the “transactional” nature of Atomic, but 
we’re finding there’s no real “commit”.  When we need to change something in 
the OS itself, we end up stuck with a “hotfix” flagged deployment that we can’t 
really “commit” to clear the flag, which leaves us unable to make follow-on 
transactional changes.

We are also in the process of migrating our Ansible-based platform 
customization and software-load (i.e., Docker containers managed via Docker 
Compose) which seems somewhat unnatural in the Atomic world.

The docs seem pretty clearly focused on using Atomic as the OS for a cluster, 
but our standalone deployments seem to leave us with a lot of questions.  I 
think the following list is a reasonable summary of the big questions we 
haven’t really figured out yet:


  *   Is there a “commit” that would allow us to commit the current “hotfix” to 
the current deployment?
  *   Once in “hotfix” mode, is there anyway to get out of it that doesn’t toss 
the existing changes?
  *   All of our appliances start with a common base image.  What’s the 
recommended approach for changing kernel arguments so they appear in the common 
base image?
  *   What am I not telling you that would be helpful in this case?

Note that we’re both invested and committed to using Atomic, and we’d like to 
offer up a resource to help flesh out the “standalone” side of the 
documentation so we both have some agreed upon contract between users and core 
devs, as well as to better enable new users with this use case.

Thanks in advance –

Shane O.

P.S.  If you’re in the Raleigh, NC area, I’d happily buy lunch or a beer (or 
both!) for a chance to pick someone’s brain on some of these issues.


This e-mail message, including any attachments and previous email messages sent 
with it, contains CONFIDENTIAL and PROPRIETARY information of Cree, Inc. or its 
subsidiaries and may be legally PRIVILEGED. You may not use, disclose, 
reproduce or distribute such information without Cree's authorization. If you 
have received this message in error, please notify the sender immediately and 
permanently delete the original message, its attachments and any copies thereof.