[solo5] KVM Microvm
Hi All I found this kinda interesting and relavent to running virtio unikernels, thought i would let others known. https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg06211.html Cheers Adam
Re: [solo5] OpenBSD and spt
‐‐‐ Original Message ‐‐‐ On Monday, April 29, 2019 8:52 PM, Martin Lucina wrote: > Hi Adam, > > On Friday, 26.04.2019 at 06:00, Adam Steen wrote: > > > Good Afternoon all > > Is there any appetite for a cross platform support in solo5 SPT? > > It's certainly something that's possible, but the state of the spt code in > general is a bit too raw to consider abstracting it to target different > host kernels just yet. Ok, easy enough. > > Also, I don't think there is as much of a pressing need for spt as on Linux > -- I'd expect most people running OpenBSD on amd64 to be doing so on bare > metal, i.e. with access to the CPU virtualization hardware, where you can > just run hvt. I was thinking a use can on virtualized hardware (ie https://openbsd.amsterdam/) or even non-amd64 hardware > > > The system call restriction would be very easy, and I expect the other code > > to compile with very little changes. > > What would you use for implementing the syscall restrictions? The > granularity of pledge(2) is different to that of seccomp/BPF... OpenBSD could not be as strict as Linux/seccomp/BPF, but also a lot less complex, using pledge with a promise of "stdio" (see below), this would also remove file system access. > > -mato Cheers Adam Quote from the pledge man page(https://man.openbsd.org/pledge) "stdio" The following system calls are permitted. sendto(2) is only permitted if its destination socket address is NULL. As a result, all the expected functionalities of libc stdio work. clock_getres(2), clock_gettime(2), close(2), closefrom(2), dup(2), dup2(2), dup3(2), fchdir(2), fcntl(2), fstat(2), fsync(2), ftruncate(2), getdents(2), getdtablecount(2), getegid(2), getentropy(2), geteuid(2), getgid(2), getgroups(2), getitimer(2), getlogin(2), getpgid(2), getpgrp(2), getpid(2), getppid(2), getresgid(2), getresuid(2), getrlimit(2), getrtable(2), getsid(2), getthrid(2), gettimeofday(2), getuid(2), issetugid(2), kevent(2), kqueue(2), lseek(2), madvise(2), minherit(2), mmap(2), mprotect(2), mquery(2), munmap(2), nanosleep(2), pipe(2), pipe2(2), poll(2), pread(2), preadv(2), pwrite(2), pwritev(2), read(2), readv(2), recvfrom(2), recvmsg(2), select(2), sendmsg(2), sendsyslog(2), sendto(2), setitimer(2), shutdown(2), sigaction(2), sigprocmask(2), sigreturn(2), socketpair(2), umask(2), wait4(2), write(2), writev(2)
[solo5] OpenBSD and spt
Good Afternoon all Is there any appetite for a cross platform support in solo5 SPT? The system call restriction would be very easy, and I expect the other code to compile with very little changes. Would like to hear your thoughts? Cheers Adam
Re: [solo5] OpenBSD Support
‐‐‐ Original Message ‐‐‐ On Thursday, March 28, 2019 5:36 PM, Martin Lucina wrote: > On Wednesday, 20.03.2019 at 10:29, Martin Lucina wrote: > > > Hi Adam, > > On Wednesday, 20.03.2019 at 05:54, Adam Steen wrote: > > > > > Hi All > > > With the soon to be released OpenBSD 6.5, having Opam 2.0.3 and OCaml > > > 4.07.1 as packages, I thought i would open discussion about which release > > > of OpenBSD should be supported by Solo5. > > > > That decision is mainly up to you as the de facto maintainer of the OpenBSD > > support. > > > > > I was thinking it should be the latest stable release, ie as of the > > > release of 6.5, Not Current, it just seems too much of a moving target. > > > > Personally I'd keep it simple (and reduce the amount of work involved), and > > stick to supporting 6.5 only. > > Actually, we should continue to support 6.4 as that now has CI (added this > week, finally!) and then 6.5 only after it's released. It's impossible for > me to track -current for CI. > > IOW, we should stive to support the current releases (N, N - 1). > > -mato Hi Mato I agree, the goal should be to support "Only the two most recent OpenBSD" as per [1], or as you put it (N, N - 1). [1] https://www.openbsd.org/faq/faq5.html#Flavors Cheers Adam
[solo5] OpenBSD Support
Hi All With the soon to be released OpenBSD 6.5, having Opam 2.0.3 and OCaml 4.07.1 as packages, I thought i would open discussion about which release of OpenBSD should be supported by Solo5. I was thinking it should be the latest stable release, ie as of the release of 6.5, Not Current, it just seems too much of a moving target. I would greatly appreciate your input. Cheers Adam
Re: [solo5] Is dumpcore enabled
‐‐‐ Original Message ‐‐‐ On August 16, 2018 6:11 PM, Martin Lucina wrote: > On Wednesday, 15.08.2018 at 05:00, Adam Steen wrote: > > > Hi Martin > > I have complete the pull request [1], and was looking for further > > discussion from anyone who is interested. > > Thanks, I'll look into it, but it might take a week or so, currently > concentrating on the renaming/refactor for #172. > > It might be easier to do this after that is done as all the internal API > names will change. > > Cheers, > > -mato Hi Mato I think your right, lets do it after the rename/refactor, the code is there and i can use it, and i more things than time that i need to do. Cheers Adam
Re: [solo5] Is dumpcore enabled
Hi Martin I have complete the pull request [1], and was looking for further discussion from anyone who is interested. Cheers Adam [1] https://github.com/Solo5/solo5/pull/271 ‐‐‐ Original Message ‐‐‐ On 13 August 2018 3:05 PM, Adam Steen wrote: > ‐‐‐ Original Message ‐‐‐ > On 13 August 2018 1:47 AM, Martin Lucina mar...@lucina.net wrote: > > > Hi Adam, > > apologies for the delayed reply, I've been on vacation. > > On Sunday, 22.07.2018 at 22:36, Adam Steen wrote: > > > > > Hi All > > > After a quick discussion with Hannes and attempting to implement coredump > > > on OpenBSD[1], i found there was no easy way to determine if use_coredump > > > was true. ie coredump was enabled., > > > I had to remove a tight pledge, privilege drop and chroot just to enable > > > coredump, i wasn't sure it was worth it. Hannes suggested turning the > > > security changes on or off if coredump was enabled, but we both couldn't > > > find a easy way to do this. > > > So i am posting here to get the discussion started about, what we can do > > > to make this easier. > > > The only things i could come up with, with the current code, was to > > > disable the security if the dumpcore module was compile in (available). > > > see [2] and [3]. > > > I wanted to raise this discussion not around my coredump code but about > > > determining which module(s) are enabled in the ukvm main/system code > > > > This is a good question. With the current model of determining modules to > > enable at compile time, I would use the same approach for determining > > whether or not to "drop privileges" (not sure what best to call this > > functionality, suggestions please?). > > Specifically: > > > > 1. Add a compile-time #define, e.g. UKVM_DROP_PRIVILEGES. In a suitable > > header, say ukvm/ukvm.h since that is included by all modules, define > > this > > to 1 if not already defined. I.e. default to dropping privileges. > > > > 2. ukvm-configure can then manually add -DUKVM_DROP_PRIVILEGES=0 to CFLAGS > > if dumpcore has been requested. > > > > 3. If UKVM_DROP_PRIVILEGES=1, you get the current behaviour in your code. > > > > 4. If UKVM_DROP_PRIVILEGES=0, privilege dropping is disabled AND ukvm > > prints a stern warning to this effect at startup, including something > > about > > not being recommended for production, etc. > > Separately from this, and with a view to adding some amount of privilege > > dropping by default on other systems besides OpenBSD, I think that: > > > > 5. The privilege dropping code should be moved into its own top-level > > function, e.g. ukvm_hv_drop_privileges(), which goes into > > ukvm_hv_.c. > > > > 6. This function is clearly called from ukvm/ukvm_main.c, just before > > entering the VCPU loop(?). This would also be where the #if printing the > > warning if disabled (see (4) above) goes. > > > > > > As for what privileges should exactly be dropped on other OSes by default, > > I need to think about that a bit more and will follow up during the week. > > However, this should be enough to get you started. > > Would this approach work for you? Any other opinions? > > Cheers, > > -mato > > this approach works really well, i like it! > > I hope to have something along these lines this week! > > Cheers > Adam
Re: [solo5] Is dumpcore enabled
‐‐‐ Original Message ‐‐‐ On 13 August 2018 1:47 AM, Martin Lucina wrote: > Hi Adam, > > apologies for the delayed reply, I've been on vacation. > > On Sunday, 22.07.2018 at 22:36, Adam Steen wrote: > > > Hi All > > After a quick discussion with Hannes and attempting to implement coredump > > on OpenBSD[1], i found there was no easy way to determine if use_coredump > > was true. ie coredump was enabled., > > I had to remove a tight pledge, privilege drop and chroot just to enable > > coredump, i wasn't sure it was worth it. Hannes suggested turning the > > security changes on or off if coredump was enabled, but we both couldn't > > find a easy way to do this. > > So i am posting here to get the discussion started about, what we can do to > > make this easier. > > The only things i could come up with, with the current code, was to disable > > the security if the dumpcore module was compile in (available). see [2] and > > [3]. > > I wanted to raise this discussion not around my coredump code but about > > determining which module(s) are enabled in the ukvm main/system code > > This is a good question. With the current model of determining modules to > enable at compile time, I would use the same approach for determining > whether or not to "drop privileges" (not sure what best to call this > functionality, suggestions please?). > > Specifically: > > 1. Add a compile-time #define, e.g. UKVM_DROP_PRIVILEGES. In a suitable > header, say ukvm/ukvm.h since that is included by all modules, define this > to 1 if not already defined. I.e. default to dropping privileges. > > 2. ukvm-configure can then manually add -DUKVM_DROP_PRIVILEGES=0 to CFLAGS > if dumpcore has been requested. > > 3. If UKVM_DROP_PRIVILEGES=1, you get the current behaviour in your code. > 4. If UKVM_DROP_PRIVILEGES=0, privilege dropping is disabled AND ukvm > prints a stern warning to this effect at startup, including something > about > not being recommended for production, etc. > > Separately from this, and with a view to adding some amount of privilege > dropping by default on other systems besides OpenBSD, I think that: > > 5. The privilege dropping code should be moved into its own top-level > function, e.g. ukvm_hv_drop_privileges(), which goes into ukvm_hv_.c. > > > 2. This function is clearly called from ukvm/ukvm_main.c, just before > entering the VCPU loop(?). This would also be where the #if printing the > warning if disabled (see (4) above) goes. > > As for what privileges should exactly be dropped on other OSes by default, > I need to think about that a bit more and will follow up during the week. > However, this should be enough to get you started. > > Would this approach work for you? Any other opinions? > > Cheers, > > -mato this approach works really well, i like it! I hope to have something along these lines this week! Cheers Adam
[solo5] Is dumpcore enabled
Hi All After a quick discussion with Hannes and attempting to implement coredump on OpenBSD[1], i found there was no easy way to determine if use_coredump was true. ie coredump was enabled., I had to remove a tight pledge, privilege drop and chroot just to enable coredump, i wasn't sure it was worth it. Hannes suggested turning the security changes on or off if coredump was enabled, but we both couldn't find a easy way to do this. So i am posting here to get the discussion started about, what we can do to make this easier. The only things i could come up with, with the current code, was to disable the security if the dumpcore module was compile in (available). see [2] and [3]. I wanted to raise this discussion not around my coredump code but about determining which module(s) are enabled in the ukvm main/system code Cheers Adam [1] https://github.com/adamsteen/solo5/tree/dumpcore (currently not working correctly.) [2] https://github.com/adamsteen/solo5/blob/dumpcore/ukvm/ukvm_hv_openbsd_x86_64.c#L177 [3] https://github.com/adamsteen/solo5/blob/dumpcore/ukvm/ukvm_hv_openbsd_x86_64.c#L92-L99
Re: [solo5] Solo5, MireageOS and Write and Exec Memory
Well now i feel like a real goose! I cleaned out everything and started a fresh. | ___| __| _ \ | _ \ __ \ \__ \ ( | | ( | ) | /\___/ _|\___// Solo5: Memory map: 512 MB addressable: Solo5: unused @ (0x0 - 0xf) Solo5: text @ (0x10 - 0x21efff) Solo5: rodata @ (0x21f000 - 0x24efff) Solo5: data @ (0x24f000 - 0x30cfff) Solo5: heap >= 0x30d000 < stack < 0x2000 2018-06-22 08:34:29 -00:00: INF [application] hello 2018-06-22 08:34:30 -00:00: INF [application] hello 2018-06-22 08:34:31 -00:00: INF [application] hello 2018-06-22 08:34:32 -00:00: INF [application] hello Solo5: solo5_exit(0) called No warning, no problems. it was as simple as opam install mirage -y&& \ git clone git://github.com/mirage/mirage-skeleton.git $HOME/devl/mirage-skeleton && \ cd $HOME/devl/mirage-skeleton/tutorial/hello && \ mirage configure -t ukvm && \ gmake depends && \ gmake and then doas ukvm-bin hello.ukvm There is a patch on po...@openbsd.org for Ocaml 4.06 and OPAM 2.00rc2, I think once that goes in, it might be time to announce MirageOS on OpenBSD to m...@openbsd.org. Sorry about wasting your time mato, i have no explanation as to why it failed or why it worked again, the only difference was a clean out of .opam and 24-36hours. Cheers Adam ‐‐‐ Original Message ‐‐‐ On June 21, 2018 8:03 PM, Martin Lucina wrote: > > > On Thursday, 21.06.2018 at 07:46, Adam Steen wrote: > > > > > Mirage-Skeleton Hello World Tutorial, I now find this causes as " > > > > > > > > mprotect W^X violation" on OpenBSD. > > > > > > That's err, odd. Do you get a "Warning: phdr[N] requests WRITE and EXEC > > > > > > permissions" when running ukvm-bin? > > > > Yes. > > Right, so your ELF is requesting such a segment, in which case the failure > > is expected (without manually disabling W^X for the ukvm-bin). > > > > When you write "causes a mprotect W^X violation", how does that actually > > > > > > manifest itself at run-time? Does ukvm-bin get killed? Or some other way? > > > > ukvm-bin gets killed by the kernel. > > > > "ukvm-bin: hello.ukvm: Warning: phdr[0] requests WRITE and EXEC permissions" > > > > "ukvm-bin: hello.ukbm: Not Supported" > > The second error would be from the mprotect() returning ENOTSUP, which is > > due to the W^X policy, and ukvm-bin is correctly aborting in that case. > > > Looks like I will have to dig a little deeper, I do recall something like > > this happening soon after i got things working on a custom kernel. I am > > awol again next week, but will try to nail down exactly what caused the > > issue. > > > > Thanks for the confirmation about linux and a starting point with "readelf > > -l" > > As I wrote initially, this looks like something in the MirageOS build on > > OpenBSD is generating a segment with W & X on it. You can double-confirm by > > e.g. building the same unikernel on Linux and seeing if it runs on OpenBSD. > > If it changed recently then it's possible that either a) you were > > previously running a kernel that did not enforce this or b) the build > > toolchain changed and/or due to a) you did not notice that it's producing W > > & X segments in the first place. > > I'd ask around on the OpenBSD lists and/or check the various clang / ld > > manual pages for any options that might affect this. > > -mato
Re: [solo5] Solo5, MireageOS and Write and Exec Memory
Hi Mato See inlines below ‐‐‐ Original Message ‐‐‐ On June 21, 2018 6:05 PM, Martin Lucina wrote: > > > Hi Adam, > > On Wednesday, 20.06.2018 at 22:07, Adam Steen wrote: > > > Hi All > > > > As some of you know I have been working to get MirageOS/Solo5 working on > > > > OpenBSD. > > > > As of Friday of last week, I though i had at least achieved this, but > > > > after running an end to end test with released version the of > > > > Mirage-Skeleton Hello World Tutorial, I now find this causes as " > > > > mprotect W^X violation" on OpenBSD. > > That's err, odd. Do you get a "Warning: phdr[N] requests WRITE and EXEC > > permissions" when running ukvm-bin? Yes. > > I know Solo5 does not issue any mprotect requests with WRITE and EXEC > > > > permissions, but something in MirageOS does. I have be testing building > > > > the Hello World example for a while now without any problems of this > > > > nature, i am not sure where to start look at what changed. > > > > I can permit W^X memory on OpenBSD with a change to the Solo5 configure > > > > script and a file system setting, but this has now missed the boat for > > > > this release and would prefer not to do this. > > That would be just papering over the problem. Your right, it should be sorted correctly. > As for fixes, it'll be while > > yet before the post 0.3.0 renaming and restructuring gets done, I can > > easily cut a point release before then for trivial fixes (no ABI changes). I will have to see what i can track down. > > > Any tips on where i should look? > > When you write "causes a mprotect W^X violation", how does that actually > > manifest itself at run-time? Does ukvm-bin get killed? Or some other way? ukvm-bin gets killed by the kernel. "ukvm-bin: hello.ukvm: Warning: phdr[0] requests WRITE and EXEC permissions" "ukvm-bin: hello.ukbm: Not Supported" and in dmesg "ukvm-bin(65577): mprotect W^X violation" > I've looked at the build products (on Linux) for both the Solo5 standalone > > tests and the mirage-skeleton tutorial/hello unikernel with "readelf -l", > > and I don't see any phdrs asking for W and X protections at the same time, > > so my guess would be something in the OpenBSD build is acting up (again). > > -mato Looks like I will have to dig a little deeper, I do recall something like this happening soon after i got things working on a custom kernel. I am awol again next week, but will try to nail down exactly what caused the issue. Thanks for the confirmation about linux and a starting point with "readelf -l" Adam
[solo5] Solo5, MireageOS and Write and Exec Memory
Hi All As some of you know I have been working to get MirageOS/Solo5 working on OpenBSD. As of Friday of last week, I though i had at least achieved this, but after running an end to end test with released version the of Mirage-Skeleton Hello World Tutorial, I now find this causes as " mprotect W^X violation" on OpenBSD. I know Solo5 does not issue any mprotect requests with WRITE and EXEC permissions, but something in MirageOS does. I have be testing building the Hello World example for a while now without any problems of this nature, i am not sure where to start look at what changed. I can permit W^X memory on OpenBSD with a change to the Solo5 configure script and a file system setting, but this has now missed the boat for this release and would prefer not to do this. Any tips on where i should look? For anyone using OpenBSD wanting to test things you can follow the steps needed to get a working Solo5 + OPAM 2 installation on a fresh OpenBSD 6.3 install at https://github.com/Solo5/solo5/issues/206#issuecomment-386415256. I would run "opam init -a --compiler=4.06.1" to get the correct compiler version straight up. And a big thanks for Mato, Hannes and the rest of the Solo5 team in helping me get things going. Cheers Adam ps Should i post this on the MirageOS mailings instead?
