Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

what about requiring sudo to do nsenter? (even when using runc rootless)



On Mon, Mar 5, 2018 at 4:09 PM, Giuseppe Scrivano 
wrote:

> Muayyad AlSadi  writes:
>
> > when using runc
> >
> > $ mypid=`runc list | tail -n 1 | awk '{print $2}'`
> > $ nsenter -a -t $mypid /bin/sh
> > nsenter: reassociate to namespace 'ns/cgroup' failed: Operation not
> permitted
> > $ sudo nsenter -a -t $mypid /bin/sh
> > # worked fine
> >
> > but when using bwraps
> >
> > $ mypid=`bwrap-oci list | tail -n 1 | awk '{print $2}'
> > $ nsenter -a -t $mypid /bin/sh
> > nsenter: reassociate to namespace 'ns/net' failed: Operation not
> permitted
> > $ sudo nsenter -a -t $mypid /bin/sh
> > nsenter: failed to execute /bin/sh: No such file or directory
>
> I guess that is an issue in bwrap as it internally uses chroot instead
> of a pivot_root.  This PR should probably fix the problem you are
> seeing:
>
>   https://github.com/projectatomic/bubblewrap/pull/256
>
> Giuseppe
>

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Giuseppe Scrivano]

Muayyad AlSadi  writes:

> when using runc
>
> $ mypid=`runc list | tail -n 1 | awk '{print $2}'`
> $ nsenter -a -t $mypid /bin/sh
> nsenter: reassociate to namespace 'ns/cgroup' failed: Operation not permitted
> $ sudo nsenter -a -t $mypid /bin/sh
> # worked fine
>
> but when using bwraps
>
> $ mypid=`bwrap-oci list | tail -n 1 | awk '{print $2}'
> $ nsenter -a -t $mypid /bin/sh
> nsenter: reassociate to namespace 'ns/net' failed: Operation not permitted
> $ sudo nsenter -a -t $mypid /bin/sh
> nsenter: failed to execute /bin/sh: No such file or directory

I guess that is an issue in bwrap as it internally uses chroot instead
of a pivot_root.  This PR should probably fix the problem you are
seeing:

  https://github.com/projectatomic/bubblewrap/pull/256

Giuseppe

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

when using runc

$ mypid=`runc list | tail -n 1 | awk '{print $2}'`
$ nsenter -a -t $mypid /bin/sh
nsenter: reassociate to namespace 'ns/cgroup' failed: Operation not
permitted
$ sudo nsenter -a -t $mypid /bin/sh
# worked fine

but when using bwraps

$ mypid=`bwrap-oci list | tail -n 1 | awk '{print $2}'
$ nsenter -a -t $mypid /bin/sh
nsenter: reassociate to namespace 'ns/net' failed: Operation not permitted
$ sudo nsenter -a -t $mypid /bin/sh
nsenter: failed to execute /bin/sh: No such file or directory


why do I need to be root to join using nsenter with runc
and why bwraps failed even if I'm root










On Mon, Mar 5, 2018 at 1:23 PM, Giuseppe Scrivano 
wrote:

> Muayyad AlSadi  writes:
>
> > it seems there is no bwrap-oci exec and nsenter does not work as regular
> user.
> >
> > how to enter an existing user name space just like "runc exec redis
> /bin/sh" using bubble wrap or nsenter?
>
> exec is not implemented yet.  The easiest way to workaround this
> limitation is to use directly "nsenter -a".
>
> Regards,
> Giuseppe
>

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Giuseppe Scrivano]

Muayyad AlSadi  writes:

> it seems there is no bwrap-oci exec and nsenter does not work as regular user.
>
> how to enter an existing user name space just like "runc exec redis /bin/sh" 
> using bubble wrap or nsenter?

exec is not implemented yet.  The easiest way to workaround this
limitation is to use directly "nsenter -a".

Regards,
Giuseppe

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

it seems there is no bwrap-oci exec and nsenter does not work as regular
user.

how to enter an existing user name space just like "runc exec redis /bin/sh"
using bubble wrap or nsenter?


On Sun, Feb 25, 2018 at 10:58 PM, Muayyad AlSadi  wrote:

> > is this still broken with my PR?
>
> no, your PR and branch works fine, please merge it
>
> > if you are interested to put this blog post in the perspective of how
> the atomic CLI works and explains its internals as you did, I can help you
> with the review and we could publish it on: http://www.projectatomic.io/bl
> og/.
> > What do you think?
>
> I'm in.
>
>
>
> On Sun, Feb 25, 2018 at 7:41 PM, Giuseppe Scrivano 
> wrote:
>
>> Muayyad AlSadi  writes:
>>
>> > here is my blog post
>> >
>> > https://bcksp.blogspot.com/2018/02/diy-docker-using-skopeoos
>> treerunc.html
>>
>> if you are interested to put this blog post in the perspective of how
>> the atomic CLI works and explains its internals as you did, I can help
>> you with the review and we could publish it on:
>> http://www.projectatomic.io/blog/.
>>
>> What do you think?
>>
>> Thanks,
>> Giuseppe
>>
>
>

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Giuseppe Scrivano]

Muayyad AlSadi  writes:

> here is my blog post
>
> https://bcksp.blogspot.com/2018/02/diy-docker-using-skopeoostreerunc.html

if you are interested to put this blog post in the perspective of how
the atomic CLI works and explains its internals as you did, I can help
you with the review and we could publish it on: 
http://www.projectatomic.io/blog/.

What do you think?

Thanks,
Giuseppe

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Giuseppe Scrivano]

Muayyad AlSadi  writes:

>> Please use the original config.json file you get with 'runc spec --rootless' 
>> and change only the process/args there.
>
> that did not work,

is this still broken with my PR?

Giuseppe

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

> Please use the original config.json file you get with 'runc spec
--rootless' and change only the process/args there.

that did not work,

> that won't work, you need to specify the mounts.  Have you tried with
bwrap-oci from the PR I've opened?

I'm using this

$ rpm -q bwrap-oci
bwrap-oci-0.2-1.fc27.x86_64

your PR and branch works fine





On Sun, Feb 25, 2018 at 4:29 PM, Giuseppe Scrivano 
wrote:

>
> Muayyad AlSadi  writes:
>
> > no, it did not work for me
> >
> > I've removed the entire mount section
> >
> > "mounts": [ ],
>
> that won't work, you need to specify the mounts.  Have you tried
> with bwrap-oci from the PR I've opened?
>
> Please use the original config.json file you get with 'runc spec
> --rootless' and change only the process/args there.
>
> Regards,
> Giuseppe
>

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Giuseppe Scrivano]

Muayyad AlSadi  writes:

> no, it did not work for me
>
> I've removed the entire mount section
>
> "mounts": [ ],

that won't work, you need to specify the mounts.  Have you tried
with bwrap-oci from the PR I've opened?

Please use the original config.json file you get with 'runc spec
--rootless' and change only the process/args there.

Regards,
Giuseppe

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

no, it did not work for me

I've removed the entire mount section

"mounts": [ ],

I tried to only remove the sys/none item in mounts,

it got stuck (no output, no error message and on another terminal it would
be running)

the following

bwrap-oci --dry-run run delme

gives

/usr/bin/bwrap --userns-block-fd FD --as-pid-1 --die-with-parent --bind
rootfs / --unshare-pid --unshare-ipc --unshare-uts --unshare-user
--unshare-user --cap-drop ALL --cap-add CAP_KILL --cap-add
CAP_NET_BIND_SERVICE --cap-add CAP_AUDIT_WRITE --chdir / --setenv PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin --setenv TERM
xterm --uid 0 --gid 0 --proc /proc --dev /dev --bind /dev/pts /dev/pts
--tmpfs /dev/shm --mqueue /dev/mqueue --tmpfs /tmp --dev-bind /dev/tty
/dev/tty --hostname runc --block-fd FD --sync-fd FD --info-fd FD --bind
/dev/null /proc/kcore --bind /dev/null /proc/latency_stats --bind /dev/null
/proc/timer_list --bind /dev/null /proc/timer_stats --bind /dev/null
/proc/sched_debug --bind /dev/null /sys/firmware --bind /dev/null
/proc/scsi --ro-bind /proc/asound /proc/asound --ro-bind /proc/bus
/proc/bus --ro-bind /proc/fs /proc/fs --ro-bind /proc/irq /proc/irq
--ro-bind /proc/sys /proc/sys --ro-bind /proc/sysrq-trigger
/proc/sysrq-trigger --remount-ro / sh

which does not work but the following words fine

/usr/bin/bwrap --as-pid-1 --die-with-parent --bind rootfs / --unshare-pid
--unshare-ipc --unshare-uts --unshare-user --unshare-user --cap-drop ALL
--cap-add CAP_KILL --cap-add CAP_NET_BIND_SERVICE --cap-add CAP_AUDIT_WRITE
--chdir / --setenv PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin --setenv TERM
xterm --uid 0 --gid 0 --proc /proc --dev /dev --bind /dev/pts /dev/pts
--tmpfs /dev/shm --mqueue /dev/mqueue --tmpfs /tmp --dev-bind /dev/tty
/dev/tty --hostname runc --remount-ro / sh

the config is attached





On Sun, Feb 25, 2018 at 2:01 PM, Giuseppe Scrivano 
wrote:

> Hi Muayyad,
>
> Muayyad AlSadi  writes:
>
> > here is my blog post
> >
> > https://bcksp.blogspot.com/2018/02/diy-docker-using-
> skopeoostreerunc.html
>
> That is definitely a great blog post!  It is a very good explanation of
> how the atomic CLI works for a non root user.
>
>
> > the error in "bwrap-oci run"
> > bwrap-oci: unknown mount type none
> > was because of type none in /sys
> >
> > "mounts": [
> > ...
> > {
> > "destination": "/sys",
> > "type": "none",
> > "source": "/sys",
> > "options": [
> > "rbind",
> > "nosuid",
> > "noexec",
> > "nodev",
> > "ro"
> > ]
> > }
> >
> > but removing it did not solve the problem
>
> The issue you reported is a bug in bwrap-oci.  It fails with an error
> caused by the '"type" : "none"' generated by .runc spec --rootless.
>
> Could you please try if this PR solves the problem for you?
>
>   https://github.com/projectatomic/bwrap-oci/pull/17
>
> Another option is to change "none" to "bind" in the configuration file.
>
> In general bwrap-oci is more tolerant than runc with the config.json
> configuration.  bwrap-oci takes the freedom of adding the user namespace
> even if it is not specified and handle the users mapping inside of the
> container (if you need more than one user mapped please take a look at
> /etc/subuid and /etc/subgid).  It is designed this way so that the
> configuration that works for a system container could to some extend be
> used by a non root user in a seamless way.
>
> You should be fine to run the container with the config.json file you
> get with "runc spec" without the "--rootless" option.
>
> Please let me know if this works for you.
>
> Regards,
> Giuseppe
>


config.json
Description: application/json

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Giuseppe Scrivano]

Hi Muayyad,

Muayyad AlSadi  writes:

> here is my blog post
>
> https://bcksp.blogspot.com/2018/02/diy-docker-using-skopeoostreerunc.html

That is definitely a great blog post!  It is a very good explanation of
how the atomic CLI works for a non root user.


> the error in "bwrap-oci run"
> bwrap-oci: unknown mount type none
> was because of type none in /sys
>
> "mounts": [
> ...
> {
> "destination": "/sys",
> "type": "none",
> "source": "/sys",
> "options": [
> "rbind",
> "nosuid",
> "noexec",
> "nodev",
> "ro"
> ]
> }
>
> but removing it did not solve the problem

The issue you reported is a bug in bwrap-oci.  It fails with an error
caused by the '"type" : "none"' generated by .runc spec --rootless.

Could you please try if this PR solves the problem for you?

  https://github.com/projectatomic/bwrap-oci/pull/17

Another option is to change "none" to "bind" in the configuration file.

In general bwrap-oci is more tolerant than runc with the config.json
configuration.  bwrap-oci takes the freedom of adding the user namespace
even if it is not specified and handle the users mapping inside of the
container (if you need more than one user mapped please take a look at
/etc/subuid and /etc/subgid).  It is designed this way so that the
configuration that works for a system container could to some extend be
used by a non root user in a seamless way.

You should be fine to run the container with the config.json file you
get with "runc spec" without the "--rootless" option.

Please let me know if this works for you.

Regards,
Giuseppe

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

here is my blog post

https://bcksp.blogspot.com/2018/02/diy-docker-using-skopeoostreerunc.html

the error in "bwrap-oci run"
bwrap-oci: unknown mount type none
was because of type none in /sys

"mounts": [
...
{
"destination": "/sys",
"type": "none",
"source": "/sys",
"options": [
"rbind",
"nosuid",
"noexec",
"nodev",
"ro"
]
}

but removing it did not solve the problem



On Fri, Feb 23, 2018 at 1:49 AM, Muayyad AlSadi  wrote:

> after that, the following worked
>
> cd cont1
> runc spec
> runc run myname
>
> I also tried "runc spec --rootless" and it worked but bwrap-oci did not
>
> $ bwrap-oci run
> bwrap-oci: unknown mount type none
>
>
>
> On Fri, Feb 23, 2018 at 1:33 AM, Muayyad AlSadi  wrote:
>
>> ostree checkout ociimage/nginx_3Alatest cont1
>> cat cont1/manifest.json | jq '.layers[]|.digest' | sed -re 's/"//g' | cut
>> -d ':' -f 2 | while read a; do echo ostree checkout --union ociimage/$a
>> cont1/rootfs; done
>>
>> what's next?
>>
>>
>> On Fri, Feb 23, 2018 at 12:18 AM, Muayyad AlSadi 
>> wrote:
>>
>>> hi,
>>>
>>> I'm running fedora as regular user
>>> and I wonder how can I use skopeo+ostree+bwrap-oci to run a docker image
>>> using bwrap-oci having files stored as ostree
>>>
>>> $ mkdir ostree
>>> $ cd ostree
>>> $ ostree init --mode=bare-user --repo=$PWD
>>> $ skopeo copy docker://redis:alpine ostree:redis@$PWD
>>> $ skopeo copy docker://nginx:alpine ostree:nginx@$PWD
>>> $ skopeo copy docker://busybox:alpine ostree:busybox@$PWD
>>> $ ostree refs
>>> $ ostree ls ociimage/redis_3Alatest
>>> $ ostree checkout ociimage/nginx_3Alatest cont1
>>> $ cd cont1
>>> $ bwrap-oci -c manifest.json run
>>> bwrap: --userns-block-fd requires --unshare-user
>>>
>>> so what went wrong?
>>>
>>>
>>
>

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

after that, the following worked

cd cont1
runc spec
runc run myname

I also tried "runc spec --rootless" and it worked but bwrap-oci did not

$ bwrap-oci run
bwrap-oci: unknown mount type none



On Fri, Feb 23, 2018 at 1:33 AM, Muayyad AlSadi  wrote:

> ostree checkout ociimage/nginx_3Alatest cont1
> cat cont1/manifest.json | jq '.layers[]|.digest' | sed -re 's/"//g' | cut
> -d ':' -f 2 | while read a; do echo ostree checkout --union ociimage/$a
> cont1/rootfs; done
>
> what's next?
>
>
> On Fri, Feb 23, 2018 at 12:18 AM, Muayyad AlSadi  wrote:
>
>> hi,
>>
>> I'm running fedora as regular user
>> and I wonder how can I use skopeo+ostree+bwrap-oci to run a docker image
>> using bwrap-oci having files stored as ostree
>>
>> $ mkdir ostree
>> $ cd ostree
>> $ ostree init --mode=bare-user --repo=$PWD
>> $ skopeo copy docker://redis:alpine ostree:redis@$PWD
>> $ skopeo copy docker://nginx:alpine ostree:nginx@$PWD
>> $ skopeo copy docker://busybox:alpine ostree:busybox@$PWD
>> $ ostree refs
>> $ ostree ls ociimage/redis_3Alatest
>> $ ostree checkout ociimage/nginx_3Alatest cont1
>> $ cd cont1
>> $ bwrap-oci -c manifest.json run
>> bwrap: --userns-block-fd requires --unshare-user
>>
>> so what went wrong?
>>
>>
>

Re: [atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

ostree checkout ociimage/nginx_3Alatest cont1
cat cont1/manifest.json | jq '.layers[]|.digest' | sed -re 's/"//g' | cut
-d ':' -f 2 | while read a; do echo ostree checkout --union ociimage/$a
cont1/rootfs; done

what's next?


On Fri, Feb 23, 2018 at 12:18 AM, Muayyad AlSadi  wrote:

> hi,
>
> I'm running fedora as regular user
> and I wonder how can I use skopeo+ostree+bwrap-oci to run a docker image
> using bwrap-oci having files stored as ostree
>
> $ mkdir ostree
> $ cd ostree
> $ ostree init --mode=bare-user --repo=$PWD
> $ skopeo copy docker://redis:alpine ostree:redis@$PWD
> $ skopeo copy docker://nginx:alpine ostree:nginx@$PWD
> $ skopeo copy docker://busybox:alpine ostree:busybox@$PWD
> $ ostree refs
> $ ostree ls ociimage/redis_3Alatest
> $ ostree checkout ociimage/nginx_3Alatest cont1
> $ cd cont1
> $ bwrap-oci -c manifest.json run
> bwrap: --userns-block-fd requires --unshare-user
>
> so what went wrong?
>
>

[atomic-devel] how to try combining skopeo+ostree+bwrap-oci

[Muayyad AlSadi]

hi,

I'm running fedora as regular user
and I wonder how can I use skopeo+ostree+bwrap-oci to run a docker image
using bwrap-oci having files stored as ostree

$ mkdir ostree
$ cd ostree
$ ostree init --mode=bare-user --repo=$PWD
$ skopeo copy docker://redis:alpine ostree:redis@$PWD
$ skopeo copy docker://nginx:alpine ostree:nginx@$PWD
$ skopeo copy docker://busybox:alpine ostree:busybox@$PWD
$ ostree refs
$ ostree ls ociimage/redis_3Alatest
$ ostree checkout ociimage/nginx_3Alatest cont1
$ cd cont1
$ bwrap-oci -c manifest.json run
bwrap: --userns-block-fd requires --unshare-user

so what went wrong?