Re: Snappy sprint reporty musing

[Harald Sitter]

On Tue, Jul 26, 2016 at 1:08 PM, Harald Sitter  wrote:
> Snappy right now has limited exposure as it is only available on
> Ubuntu (Unity). This is actively being worked on with main blockers
> being missing golang stacks (snapd is mostly written in Go), missing
> apparmor patches (pending upstreaming), and systems that don't use
> apparmor at all which I guess is the biggest hurdle (needs special
> kernel configurations). Overall there seems to be interest in having
> things work well everywhere, which is good of course. Additionally,
> there's also cool stuff in the works such as adding snap build support
> in the open build service (OBS).

Since there is some confusion over this: Right now you can already
install snapd on the big distros, and you can probably use snaps just
fine. The problem is that sometimes those builds are unofficial (i.e.
not from the main repositories) and they require kernels with suitable
apparmor patches to make confinement actually work, so you potentially
need untrusted/unsigned kernels to make this fly as well as on Ubuntu.
This is rapidly improving however.

For testing purposes you should be fine as long as you install snaps
with `snap install --devmode yolo.snap` as that will mostly behave
like a glorified container (do not that on some distros might actually
need their security module disabled e.g selinux on fedora). For actual
production use it's kinda meh still.

HS

Re: Snappy sprint reporty musing

[Harald Sitter]

On Tue, Jul 26, 2016 at 2:07 PM, Sebastian Kügler  wrote:
> On Tuesday, July 26, 2016 1:08:28 PM CEST Harald Sitter wrote:
>> - a store REST API (of which the reference version is the ubuntu store)
>
> So something like this exists for flatpaks as well, and it's open source? For
> snappy, we'd either have to use the ubuntu store (non-free, right?) or write
> our own from scratch?
>
> Could you expand on the distribution mechanism?

Right, so, this is actually where the two systems diverge the
strongest in philosophy IMO.

Flatpak distributes via repositories. Those are for all intents and
purposes like any old rpm/deb repo a file tree of stuff the client may
or may not want to retrieve. They are very much meant to be
distributed. So, we would have a repo, GNOME would have a repo,
LibreOffice would have a repo and the user (or her distribution) has
to add the relevant repos to their system to gain access to the
applications inside. Ultimately I think distros will have to manage a
sane default pool of repos or this is going to end in tears ;)

Snaps OTOH do not use repos but some "store" which is basically a REST
API provider to do ultimately the same as a repo would do, albeit more
"webby" as it is actually an API and not just a file tree. That by
default doesn't exclude having multiple stores, the snappy team
however sounded a lot like they want to keep it central-entity by
default. Their point being that the user shouldn't have to go to KDE,
GNOME, LO... to get access to their stuff, their stuff should simply
be in a central store that would always be enabled. It's basically the
equivalent of the Google Play store. The ubuntu store API thingy is
apparently free though [1] there also is an example simple
implementation [2]. I think the biggest problem here isn't so much the
stores concept in and of itself, it's that the current snapd only can
use one store at a time, which makes it awkward if a distributor
doesn't want to use the Ubuntu store for whatever reason.

Arguments can be made for either approach. In the end I hope we'll see
a mushed together version. Fully distributed will likely get on
people's nerves and at the same time fully centralized will probably
raise eyebrows WRT trust and control.

[1] https://github.com/snapcore/snapweb
[2] https://github.com/noise/snapstore

HS

Re: Snappy sprint reporty musing

[Sebastian Kügler]

On Tuesday, July 26, 2016 2:27:34 PM CEST Aleix Pol wrote:
> > Could you expand on the distribution mechanism?
> 
> I don't understand the question.

That's what you already did with your email, so thanks. :-)
-- 
sebas

http://www.kde.org | http://vizZzion.org

Re: Snappy sprint reporty musing

[Aleix Pol]

On Tue, Jul 26, 2016 at 2:07 PM, Sebastian Kügler  wrote:
> On Tuesday, July 26, 2016 1:08:28 PM CEST Harald Sitter wrote:
>> - a store REST API (of which the reference version is the ubuntu store)
>
> So something like this exists for flatpaks as well, and it's open source?
Note for Flatpak it's ostree repositories, not REST API's.
It's similar to flatpak repositories but not entirely. I'd assume that
the Ubuntu Store will be a tad more complex than a flatpak repository,
as they plan to have a big one for all the stuff (with a big
asterisk). For example, on the Ubuntu Store, you'd be able to have 5
firefoxes.

> For snappy, we'd either have to use the ubuntu store (non-free, right?) or 
> write our own from scratch?
2 things:
- it's seems that they want to set up stores within their instance, so
it's possible to have a custom one with some parameters.
- also it should be possible to set up a separate one.

> Could you expand on the distribution mechanism?
I don't understand the question.

Aleix

Re: Snappy sprint reporty musing

[Sebastian Kügler]

On Tuesday, July 26, 2016 1:08:28 PM CEST Harald Sitter wrote:
> - a store REST API (of which the reference version is the ubuntu store)

So something like this exists for flatpaks as well, and it's open source? For 
snappy, we'd either have to use the ubuntu store (non-free, right?) or write 
our own from scratch?

Could you expand on the distribution mechanism?

Thanks for the very interesting report!
-- 
sebas

http://www.kde.org | http://vizZzion.org

Snappy sprint reporty musing

[Harald Sitter]

Hello sweeties!

Last week Scarlett, Aleix, Matthias and I took to the Snappy sprint in
Heidelberg to discuss how to make it the most useful for KDE. I'd like
to give you an overview of what was discussed along with some
background, so we are all on the same page.

If you already know what Snappy is, you can skip the next paragraph.

Snappy is one of them fancy new bundle packages: AppImage, Flatpak,
Snappy. Out of those three Flatpak and Snappy are based on Linux
Containers/CGroups/Namespace and that whole slew of magic. Simply put
they are like docker. AppImage is not using this but seeks to have
self-exectuable self-containing LD_LIBRARY_PATH'd programs.
Snappy for the most part is 3 things:
- a store REST API (of which the reference version is the ubuntu store)
- a daemon on the system (snapd, imagine it like dockerd)
- a .snap file containing the installed tree of an application
(including deps, although that's going to change soon)
Generally speaking the store bit isn't technically necessary, it is
the primary means of distribution right now though.

Shipping things users can use on Linux has been a pain in the rear
since forever and these bundles are meant to change that. As such we
as KDE should have a strong interest and presence in this field in the
hopes of shaping a future that is useful to us. After all, we are one
of the biggest source distributors, and the primary reason we don't
also offer generic binary packages of our applications is because this
never scaled and was altogether terrible to pull off from a KDE point
of view.

Unfortunately right now we are in a state of limbo with regards to
bundles. Everything sort of makes sense but none of them allows us to
do large scale building and shipping to users. Which means we need to
spread our resources a bit to get good coverage for everything. Not
spreading ourselves too thin certainly is a key concern though.

With all that in mind, let's take a look at snappy.

I think it is best if I don't get into too much detail of how exactly
snaps are built or run, but the gist of it is that snaps get
"packaged" from highlevel definitions describing what we expect to
happen (e.g. cmake should run and then we should have usr/bin/kitten
and put that in our snap). If you have seen how Flatpak does it...
describing snaps looks very similar. Using fancy kernel tech the snaps
at runtime then get isolated from the rest of the (userspace) system
allowing them to be fully compatible with any base distribution (in
theory anyway).

Snappy right now has limited exposure as it is only available on
Ubuntu (Unity). This is actively being worked on with main blockers
being missing golang stacks (snapd is mostly written in Go), missing
apparmor patches (pending upstreaming), and systems that don't use
apparmor at all which I guess is the biggest hurdle (needs special
kernel configurations). Overall there seems to be interest in having
things work well everywhere, which is good of course. Additionally,
there's also cool stuff in the works such as adding snap build support
in the open build service (OBS).

Snaps are generally built on a minimal ubuntu 16.04 core, which is the
base they then also get run against. Technically I guess that isn't
forced though (same as with Flatpak any old base could be used).

This gives us an ever so sweet advantage with Snappy. We already have
KDE neon which is based on Ubuntu 16.04! So, Scarlett and I built some
high level mass automation of snap building on top of neon's existing
deb binaries. Today we have most of the kf5 apps we have in neon also
built as snaps for testing purposes. Not all of them work of course,
but it's a start. And having them based off of the builds neon does
anyway means they are dirt cheap in both developer and build time.

Short-term this allows us to test and polish snaps. They are fairly
large but that is more because of present snappy deficiencies than
anything else (bound to be fixable very very soon).

Mid-term I think we should get to a point where we have most of our
KDE Applications available as high quality snaps and push them to the
Ubuntu store. This store should eventually one supposes be exposed
through the software center in Ubuntu (I am sure we can get something
ironed out to get discover in Kubuntu also to support it somehow).
This will probably require some policy discussion to make sure we only
release highest quality snaps as we then have direct end-user
exposure. But more on this at later time.

The long-term target of course is to not have to base off of debs at
all. A key component of this should become available soon in the form
of what Snappy calls "content sharing". It's basically a way for us to
have a "KF5" snap which gets shared by all apps that wish to use it. A
shared library bundle, if you will. This will allow us to cheaply
build applications via snappy's build tech and entirely bypass .debs.
And also minimize the size of individual application snaps. Also right
now building