Re: rumprun-bake changes for command line commands

2016-02-10 Thread Martin Lucina
On Tuesday, 09.02.2016 at 14:44, Antti Kantee wrote: > Hi, > > Per recent discussion with David, it became clear that the ability to pass > rumprun-bake config directives on the command line is desirable (IIRC it was > already proposed in the discussion last year, but there was no strong use > cas

unikernel-runner -- build and run rumprun unikernels as Docker containers

2016-02-11 Thread Martin Lucina
(Cross-posted to the http://devel.unikernel.org/ forum to catch the attention of other projects, feel free to comment here or there) Hi all, I'd like to announce 'unikernel-runner', which provides a base image you can use to build and run rumprun unikernels as Docker containers: https://github.c

Re: unikernel-runner -- build and run rumprun unikernels as Docker containers

2016-02-11 Thread Martin Lucina
On Thursday, 11.02.2016 at 17:00, Antti Kantee wrote: > On 11/02/16 15:07, Martin Lucina wrote: > >I'd like to announce 'unikernel-runner', which provides a base image you > >can use to build and run rumprun unikernels as Docker containers: > > Did you

Re: unikernel-runner -- build and run rumprun unikernels as Docker containers

2016-02-11 Thread Martin Lucina
On Thursday, 11.02.2016 at 18:26, Martin Lucina wrote: > On Thursday, 11.02.2016 at 17:00, Antti Kantee wrote: > > On 11/02/16 15:07, Martin Lucina wrote: > > >I'd like to announce 'unikernel-runner', which provides a base image you > > >can use to

Re: unikernel-runner -- build and run rumprun unikernels as Docker containers

2016-02-12 Thread Martin Lucina
On Friday, 12.02.2016 at 07:27, David Halls wrote: > > > > My intent with unikernel-runner is to use it as a platform to experiment > > with: > > > > 1. Improving Docker/unikernel integration.(e.g. native support for L2 > > connectivity so that CAP_NET_ADMIN is not required) > > 2. Use it as

Re: unikernel-runner -- build and run rumprun unikernels as Docker containers

2016-02-12 Thread Martin Lucina
On Thursday, 11.02.2016 at 16:07, Martin Lucina wrote: > My intent with unikernel-runner is to use it as a platform to experiment > with: > > 1. Improving Docker/unikernel integration.(e.g. native support for L2 > connectivity so that CAP_NET_ADMIN is not required) > 2. U

Re: [RFD] Configuration spec: "rc", "env" and "hostname"

2016-02-12 Thread Martin Lucina
On Friday, 05.02.2016 at 13:59, Antti Kantee wrote: > It's simple: things which should work in kernonly mode should be documented > (now) and implemented (future) as such. Things which can't work in kernonly > mode shouldn't work in kernonly mode. I'm going to pass on the question of what (and ho

Re: Automated builds for rumprun-packages

2016-02-17 Thread Martin Lucina
On Tuesday, 02.02.2016 at 12:40, Antti Kantee wrote: > On 02/02/16 10:48, Martin Lucina wrote: > >On Monday, 01.02.2016 at 17:41, Antti Kantee wrote: > >>Is there any progress with the automated package builds? Either they are > >>not running anymore, or all package

Re: Automated builds for rumprun-packages

2016-02-17 Thread Martin Lucina
On Wednesday, 17.02.2016 at 13:02, Dejan Strbad wrote: > What about TravisCI and build matrix? > Only drawback I can think of in that case would be need for Dockerfile per > package directory. I can try setting that up over the weekend if that > sounds acceptable. That's exactly what I'm trying to

Re: Automated builds for rumprun-packages

2016-02-19 Thread Martin Lucina
On Thursday, 18.02.2016 at 21:36, David Halls wrote: > On 17 February 2016 at 12:26, Dejan Strbad wrote: > > > Okay. Will take a look at Travis CI over the weekend and see what can be > > done. > > > > Maybe we can combine efforts. I'd like to build on a hosted CI too and > produce a set of bina

Travis CI builds for rumprun-packages available

2016-03-03 Thread Martin Lucina
Hi, I've implemented Travis CI builds for rumprun-packages. After fixing trivial issues with some packages, all[1] packages except openmp are building. See the latest results here: https://travis-ci.org/rumpkernel/rumprun-packages Due to the amount of time it takes for a build to complete (3+ ho

Dash commit 46d3c1a (in 0.5.8+) breaks BSD make

2016-03-04 Thread Martin Lucina
Hi, Commit 46d3c1a ([VAR] Sanitise environment variable names on entry) restricts exported environment variables actually set by dash on entry to those reported as valid by endofname(). While this is technically correct and matches IEEE Std 1003.1, it breaks BSD make which uses '.' as a separator

[PATCH] pci: Replace "." with "_" in exported var names

2016-03-04 Thread Martin Lucina
Two patches here, one for rumprun (the user of these variables), another for src-netbsd. Please review. Fixes rumprun issue #38 (build fails on certain distros with dash as /bin/sh). Who are the other downstream users that may be affected by this change? Related, while looking at the output from

Re: Dash commit 46d3c1a (in 0.5.8+) breaks BSD make

2016-03-04 Thread Martin Lucina
On Friday, 04.03.2016 at 14:24, Antti Kantee wrote: > On 04/03/16 14:06, Martin Lucina wrote: > >Commit 46d3c1a ([VAR] Sanitise environment variable names on entry) > >restricts exported environment variables actually set by dash on entry to > >those reported as valid by endof

Re: [PATCH] pci: Replace "." with "_" in exported var names

2016-03-04 Thread Martin Lucina
On Friday, 04.03.2016 at 16:05, Antti Kantee wrote: > On 04/03/16 15:38, Martin Lucina wrote: > >Two patches here, one for rumprun (the user of these variables), another > >for src-netbsd. Please review. Fixes rumprun issue #38 (build fails on > >certain distros with dash as

Re: Travis CI builds for rumprun-packages available

2016-03-04 Thread Martin Lucina
On Friday, 04.03.2016 at 14:57, Antti Kantee wrote: > Sounds a bit backwards to have the master list in .travis.yml. Could we > instead have a top-level Makefile, and Travis would pick up the packages > from there? Some blacklisting feature might be necessary. Unfortunately the list has to go di

Re: Travis CI builds for rumprun-packages available

2016-03-04 Thread Martin Lucina
On Friday, 04.03.2016 at 16:53, Antti Kantee wrote: > On 04/03/16 16:42, Martin Lucina wrote: > >>I think we now need a policy for packages which do not build -- I don't want > >>to perpetually advertise a broken build in case there's a package that > >>no

Re: [PATCH] pci: Replace "." with "_" in exported var names

2016-03-04 Thread Martin Lucina
On Friday, 04.03.2016 at 16:31, Antti Kantee wrote: > On 04/03/16 16:23, Martin Lucina wrote: > >On Friday, 04.03.2016 at 16:05, Antti Kantee wrote: > >>On 04/03/16 15:38, Martin Lucina wrote: > >>>Two patches here, one for rumprun (the user of these variables), anoth

Re: Travis CI builds for rumprun-packages available

2016-03-04 Thread Martin Lucina
On Friday, 04.03.2016 at 17:05, Antti Kantee wrote: > On 04/03/16 17:01, Martin Lucina wrote: > >>>IMHO the best thing to do would be to eventually change to a Dockerized > >>>build with one "build container" per package, that way at least anyone with > &

[PATCH UPDATED] Re: [PATCH] pci: Replace "." with "_" in exported var names

2016-03-08 Thread Martin Lucina
On Friday, 04.03.2016 at 18:03, Martin Lucina wrote: > > .PATH is essentially +=, i.e. you can specify it many times. However, I'm > > not sure if it will barf from an empty value or not, but in any case you can > > use an intermediate variable. > > Ok, how abo

Re: Travis CI builds for rumprun-packages available

2016-03-10 Thread Martin Lucina
On Wednesday, 09.03.2016 at 22:43, David Halls wrote: > > > > Exactly. If we build on Travis in Docker containers then we control the > > build environment since the container can use Debian > > stable/testing/whatever userland we like to build on. So we don't need to > > deal with anything Travis-

Re: Travis CI builds for rumprun-packages available

2016-03-10 Thread Martin Lucina
On Wednesday, 09.03.2016 at 22:58, David Halls wrote: > > > > The rumprun toolchain used to build the packages is built afresh from > > source (then-current git master) *for each package*. This is not ideal -- > > I'll get in touch with Travis and see if we can figure out a way to both > > optimize

Re: Travis CI builds for rumprun-packages available

2016-03-10 Thread Martin Lucina
On Wednesday, 09.03.2016 at 23:02, David Halls wrote: > > > > After discussion on the list and IRC I decided to go with a solution based > > purely on Travis as this has the least moving pieces and external > > dependencies[3]. > > > > For each build, packages are built as individual build jobs usi

Re: Travis CI builds for rumprun-packages available

2016-03-10 Thread Martin Lucina
On Wednesday, 09.03.2016 at 23:05, David Halls wrote: > > > > Success and failure notifications are sent to the > > rumprun-bui...@freelists.org mailing list, I recommend that package > > maintainers sign up there to keep track of the build status. > > > > rumpkernel-bui...@freelists.org I think?

Re: Ruby + KVM, RoR

2016-03-10 Thread Martin Lucina
On Wednesday, 02.03.2016 at 21:50, Reuben Sutton wrote: > Hi, > > I haven't tested it yet, but I believe that native gems could work in a > similar fashion to how the Node.js package works - compiling the extension > into the binary. I'll try to get a skeleton rails app running this weekend. > >

Re: Travis CI builds for rumprun-packages available

2016-03-11 Thread Martin Lucina
On Thursday, 10.03.2016 at 20:20, David Halls wrote: > > > > > > Uploading to Github requires an OUATH token with permissions to upload to > > the Github repository in .travis.yml. What are the security implications of > > this? Unless I'm missing something, anyone could trivially steal that token

Workaround for debugging 64-bit unikernels under KVM/QEMU

2016-03-15 Thread Martin Lucina
Hi all, I've managed to find a workaround for the well-known problem of GDB not being able to debug hw/x86-64 unikernels under QEMU or KVM (see GDB issue https://sourceware.org/bugzilla/show_bug.cgi?id=13984): 1. Start the unikernel, leaving it paused and waiting for GDB to connect: rumprun

Support for setting sysctl(7) nodes and rlimits in config

2016-03-19 Thread Martin Lucina
I've implemented[1] support for setting sysctl(7) nodes (both global and per-process) on the work-in-progress config branch. The syntax is fairly self-explanatory. For per-process settings, use rc[].netbsd.sysctl{}, for global settings use netbsd.sysctl{}, each of which is an object with key/value

Re: [RFD] Configuration spec: "rc", "env" and "hostname"

2016-03-19 Thread Martin Lucina
I've synched up the spec and implementation with what I think is the consensus on the "rc", "env" and "hostname" blocks: - rc[].netbsd.sysctl{} and netbsd.sysctl{} implemented as described in a separate thread. - rc[].argv[] defaults to rc[].bin if unspecified - rc[].workdir implemented, in the

Re: PHP5 rumprun package

2016-03-21 Thread Martin Lucina
Hi Vladislav, > I'm interested to run PHP5 for rumprun and tried to use an example from > rumprun packages. > Built both rumprun and rumprun php5 package from master branches. Used qemu > and kvm for running rumprun images. But faced the same issue for mysqld, > php-cgi and nginx binaries. Neithe

[RFC] Configuration specification: "net" -- Network configuration

2016-03-24 Thread Martin Lucina
Hi, given that there's been no further feedback on the "rc" thread, I'm continuing in posting parts of the work in progress configuration specification for review. This thread is for disucssing network configuration, i.e. the "net" section. The current work in progress state[1] of the specificat

Re: [RFC] Configuration specification: "net" -- Network configuration

2016-03-29 Thread Martin Lucina
Hi David, On Saturday, 26.03.2016 at 08:27, David Halls wrote: > Thanks Martin. Couple of things: > > - I want to make sure I can use an existing bridge device. I take it I can > since this spec is just saying how to configure the rumpkernel. e.g. I can > continue to use 'kvm -netdev ... -device

Re: Travis CI builds for rumprun-packages available

2016-04-14 Thread Martin Lucina
On Wednesday, 13.04.2016 at 08:07, Antti Kantee wrote: > On 13/04/16 06:28, David Halls wrote: > >>Probably, but to maximize the potential, a pointer to your work should be > >>available from somewhere else besides the mailing list archives. Suggestions > >>on that? > > > >Not yet. Is the current a

Re: src-netbsd blanket timestamp updated

2016-08-02 Thread Martin Lucina
On Tuesday, 02.08.2016 at 13:52, Antti Kantee wrote: > Hi, > > I've updated the src-netbsd timestamp to 20160728 1100UTC and pulled it up > to Rumprun and rumpctrl. All Travis tests pass. Please report issues if > you encounter them. > > The main reason for the update was fixing warnings from g

Rumprun pre-built toolchain Docker containers updated

2016-08-05 Thread Martin Lucina
Hi all, I've updated the Docker containers with pre-built Rumprun toolchains: - Added containers with Xen toolchain targets (i686 and x86_64) - Added common packages useful for building rumprun unikernels (genisoimage, xorriso, GRUB) For details, see the documentation at http://wiki.rumpkerne

Re: Is BMK mandatory when running unikernels?

2017-09-13 Thread Martin Lucina
Hi Nikhil, On Wednesday, 13.09.2017 at 13:04, nikhil ap wrote: > Thanks. I went over the FAQs and links and I couldn't find the answers for > the following questions: > > 1. What is the differnce between bmk and an hypervisor? Is bmk similar to > xen? I.E is it an hypervisor? Or is it built with

Re: Is BMK mandatory when running unikernels?

2017-09-13 Thread Martin Lucina
On Wednesday, 13.09.2017 at 15:41, nikhil ap wrote: > > bmk is a library that gets linked into a rumprun unikernel. Simplifying > > somewhat, it provides the low-level (virtual or not) hardware support and, > > among other things, an implementation of the rumpuser hypercall > > interface[1]. The di

Re: Using networking drivers as minimal components

2017-10-12 Thread Martin Lucina
Hello Jakub, On Monday, 02.10.2017 at 12:49, Jakub Jermář wrote: > I have a functional Intel gigabit (wm) rump kernel to the extent that I > can ping it and play with it as long as there is the TCP/IP stack and > sockets sitting on top of it. This is all nice, but I would like to make > it a much

Re: rumpkernel and bhyve: triple faults

2018-03-10 Thread Martin Lucina
Hi, On Friday, 09.03.2018 at 18:45, Fabian Freyer wrote: > On 6 Mar 2018, at 7:45, Fabian Freyer wrote: > > Tracking down bios_crtc_base, I find that it’s loaded in > > rumprun/platform/hw/arch/amd64/locore.S:70: > > > > /* save BIOS data area values */ > > movw BIOS_COM1_BASE, %bx > >

Re: Does rumprun support ONLY para-virtualized guests?

2018-04-04 Thread Martin Lucina
Hi, On Tuesday, 03.04.2018 at 14:02, Ajay Garg wrote: > Hi All. > > Looking at mini-os. it has code for both PV and HVM mode (controlled via > CONFIG_PARAVIRT). > > However, in rumprun codebase, there is no CONFIG_PARAVIRT. Also, all (x86) > code is the one corresponding to CONFIG_PARAVIRT code

Merging repositories (was Re: future directions for a unified rumprun)

2015-03-03 Thread Martin Lucina
po...@iki.fi said: > As step 1, I've adjusted the build scripts of -baremetal and -xen so > that they work even if buildrump.sh and rumpsrc are in ../. > > As step 2, I'll move the contents of the repos so that they'll be > non-overlapping with respect to each other. So for example -xen > will lo

Re: Merging repositories (was Re: future directions for a unified rumprun)

2015-03-03 Thread Martin Lucina
jus...@specialbusservice.com said: > Isn't that mainly a different version of submodules where the subtrees > are going to continue to exist and you may want to pull changes? Don't > we just want to do a straight merge to keep the histories and then > phase out the old repos? AFAICT this (git subt

Re: Merging repositories (was Re: future directions for a unified rumprun)

2015-03-03 Thread Martin Lucina
jus...@specialbusservice.com said: > On 3 March 2015 at 10:03, Martin Lucina wrote: > > jus...@specialbusservice.com said: > >> Isn't that mainly a different version of submodules where the subtrees > >> are going to continue to exist and you may want to pull cha

Re: Merging repositories (was Re: future directions for a unified rumprun)

2015-03-03 Thread Martin Lucina
mar...@lucina.net said: > I'll write a script to do it your way and send it to the list shortly. The following almost works. Except that the final step when I move the from-X directories to the final X causes "git mv" to segfault. Sigh. The reason for the from-X is that we already have a director

Re: Merging repositories (was Re: future directions for a unified rumprun)

2015-03-03 Thread Martin Lucina
jus...@specialbusservice.com said: > On 3 March 2015 at 10:56, Martin Lucina wrote: > > mar...@lucina.net said: > >> I'll write a script to do it your way and send it to the list shortly. > > > > The following almost works. Except that the final step when I mov

Re: future directions for a unified rumprun

2015-03-04 Thread Martin Lucina
po...@iki.fi said: > On 03/03/15 00:54, Antti Kantee wrote: > >As step 2, I'll move the contents of the repos so that they'll be > >non-overlapping with respect to each other. So for example -xen will > >look like: > > Step 2 is done. > > It might cause some test fallout, except for Travis build

Re: [PATCH 1/3] Add more stubs to build gettext

2015-03-05 Thread Martin Lucina
wei.l...@citrix.com said: > On Wed, Mar 04, 2015 at 08:14:39PM +, Antti Kantee wrote: > > On 04/03/15 19:10, Wei Liu wrote: > > >Signed-off-by: Wei Liu > > >--- > > >WEAK_STUB is needed because __sigprocmask14 is defined in both this file > > >and pthread_misc.c > > > > yes > > > > >In fact

Re: [PATCH 3/3] Add mlockall to libc stub

2015-03-05 Thread Martin Lucina
l...@liuw.name said: > Needed by QEMU. > > Signed-off-by: Wei Liu > --- > platform/xen/lib/libc_stubs.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/platform/xen/lib/libc_stubs.c b/platform/xen/lib/libc_stubs.c > index c7103b6..b1cb901 100644 > --- a/platform/xen/lib/libc_stubs.c

Re: [PATCH 2/3] Link libm to build QEMU

2015-03-05 Thread Martin Lucina
[adding -lm to endfile in specs] > > How is it needed? Why isn't pixman linking the lib in? In other words, > > what's the error without this patch? > > Not I check again, for some reason libm is not listed in LDLIBS. I think > it belongs there? > > The error is when linking QEMU, there are som

Re: [PATCH 2/3] Link libm to build QEMU

2015-03-05 Thread Martin Lucina
wei.l...@citrix.com said: > more undefined references to `sqrtf' follow > collect2: error: ld returned 1 exit status This is not normal. If I try and build a test program using sqrtf and not link it against -lm, it links fine due to GCC magically turning sqrtf into a builtin. Are you sure you're

Re: [PATCH 2/3] Link libm to build QEMU

2015-03-05 Thread Martin Lucina
wei.l...@citrix.com said: > On Thu, Mar 05, 2015 at 01:41:56PM +0100, Martin Lucina wrote: > > wei.l...@citrix.com said: > > > more undefined references to `sqrtf' follow > > > collect2: error: ld returned 1 exit status > > > > This is not normal. I

(solved) Re: [PATCH 2/3] Link libm to build QEMU

2015-03-05 Thread Martin Lucina
Following up to the list after discussion on IRC: The underlying problem is the upstream (qemu) build system. Qemu is normally built using the C++ compiler which (at least on GCC/Linux) always links with -lm. liuw's rump kernel build was using the C compiler instead which does not do that. Solut

Timekeeping and scheduling

2015-03-05 Thread Martin Lucina
Hi, While trying to get MySQL working I ran into problems with timekeeping in the rump kernel which I don't entirely understand. Basically it boils down to the following code snippet not behaving as expected: for (i= 0; i < 100; ++i) { usecs_end = get_usecs(); // sched

Re: future directions for a unified rumprun

2015-03-06 Thread Martin Lucina
po...@iki.fi said: > Step 3 is done: > > http://repo.rumpkernel.org/rumprun > > I added a before-rumprun-merge tag to rumprun-{baremetal,posix}. I > removed push access to those repos (except Justin's, since his is > implicit for all repos). I'll give it a few days before I delete > all files f

Re: Kernel Development Tutorial

2015-03-06 Thread Martin Lucina
n...@internetmachines.co.uk said: > What’s the URL? I see two > > http://wiki.netbsd.org/rumpkernel/ This is mostly historic / specific to rump kernels as used in NetBSD. > https://github.com/rumpkernel/wiki/wiki >

Re: [admin] mailing list move: archives and nntp

2015-03-06 Thread Martin Lucina
po...@iki.fi said: > The new rumpkernel-users archives are available at: > https://www.freelists.org/archive/rumpkernel-users/ > > For NNTP: > news.freelists.org, group rumpkernel-users (note: NNTP/TLS only). You might want to send this message to the old list (assuming it's still up) so that it

Re: Timekeeping and scheduling

2015-03-06 Thread Martin Lucina
po...@iki.fi said: > rump_schedule() schedules a *rump kernel cpu*, not a host thread. > sched_yield() yields the *host thread*. > > The problem is that the code you posted is essentially a busy loop. > Since the current timekeeping driver depends on the clock interrupt > running, and since all in

Re: future directions for a unified rumprun

2015-03-06 Thread Martin Lucina
po...@iki.fi said: > On 06/03/15 09:16, Martin Lucina wrote: > >After that, can we split up step 5 (actually removing duplicate parts) so > >that you don't have to do all the work? I should be able to take care of at > >least the app-tools and test/demo application

[RFC] [PATCH] Implement timecounter driver for Xen

2015-03-06 Thread Martin Lucina
form/xen/rumpxentc/rumpxen_tc.c new file mode 100644 index 000..c36cae6 --- /dev/null +++ b/platform/xen/rumpxentc/rumpxen_tc.c @@ -0,0 +1,54 @@ +/* + * Copyright (c) 2015 Martin Lucina. All Rights Reserved. + * + * Redistribution and use in source and binary forms, with or without + * modificati

Re: [RFC] [PATCH] Implement timecounter driver for Xen

2015-03-06 Thread Martin Lucina
po...@iki.fi said: > Can't you increase tc_frequency instead of returning a divided clock? Will do, should be safe even with u_int. I'm just not quite sure about how the tc stuff reacts if there is a really long busy loop where the rump kernel never gets called. > It would be better to use MODULE

[PATCH] sigwait()

2015-03-10 Thread Martin Lucina
So, MySQL calls sigwait() in a loop from a signal handling thread. Standard practice except that our sigwait is a stub, and thus mysqld wanders off into busy-loop-land. Any objections to something like the following to replace the STUB(sigtimedwait50)? diff --git a/platform/xen/lib/emul.c b/p

Re: [PATCH] sigwait()

2015-03-11 Thread Martin Lucina
po...@iki.fi said: > Wouldn't your patch result in the signal thread sleeping forever in > the scenario you describe? I'm not saying it shouldn't, it just not > what the patch immediately reads like in conjunction with your > description. Yes, that is the intention, in the first instance. In the

Migrating issues to unified rumprun repository

2015-03-12 Thread Martin Lucina
Antti, I went looking for a tool that can migrate issues from the rumprun-foo repositories to the unified rumprun repo and found this: https://github-issue-mover.appspot.com/ Related Github bug: https://github.com/holman/feedback/issues/413 Would you like me to try and use this to migrate issue

Re: [RFC] [PATCH] Implement timecounter driver for Xen

2015-03-12 Thread Martin Lucina
po...@iki.fi said: > On 06/03/15 15:04, Martin Lucina wrote: > >+RUMP_COMPONENT(RUMP_COMPONENT_KERN) > >+{ > >+tc_init(&rumpxen_tc); > >+} > > It would be better to use MODULE(). Okay, so I can do something like this, however then I don't

Re: [RFC] [PATCH] Implement timecounter driver for Xen

2015-03-12 Thread Martin Lucina
po...@iki.fi said: > On 12/03/15 10:31, Martin Lucina wrote: > >po...@iki.fi said: > >>On 06/03/15 15:04, Martin Lucina wrote: > >>>+RUMP_COMPONENT(RUMP_COMPONENT_KERN) > >>>+{ > >>>+ tc_init(&rumpxen_tc); > >>>+} > >

Re: [PATCH] sigwait()

2015-03-12 Thread Martin Lucina
sleep indefinitely if no timeout is specified. Okay to merge? commit c2155d4e06340ca66897f59698eddfdc22aabd0b Author: Martin Lucina Date: Thu Mar 12 11:53:26 2015 +0100 Implement sigtimedwait() Given that we have no signals, this just sends the calling thread to sleep and al

Signals and application shutdown (was Re: [PATCH] sigwait())

2015-03-12 Thread Martin Lucina
po...@iki.fi said: > Right, but signals is something that should be attacked critically, > not just bend over and emulating the existing concept. This whole > work is pointless if the end result is what is known as an OS ;) I do NOT want to be doing a full-blown signals implementation. What I do

Re: Upstream QEMU based stubdom and rump kernel

2015-03-18 Thread Martin Lucina
(Adding some of the Mirage folks to Cc:) wei.l...@citrix.com said: > Hi all > > I'm now working on upstream QEMU stubdom, and rump kernel seems to be a > good fit for this purpose. > > A bit background information. A stubdom is a service domain. With QEMU > stubdom we are able to run QEMU devic

Re: Upstream QEMU based stubdom and rump kernel

2015-03-18 Thread Martin Lucina
po...@rumpkernel.org said: > etfs isn't a file system, e.g. it doesn't allow listing files or > removing them, but it does give you complete control of what happens > when data is read or written for /some/path. But based on the other > posts, sounds like it might be enough for what you need. > >

Re: Upstream QEMU based stubdom and rump kernel

2015-03-18 Thread Martin Lucina
ian.campb...@citrix.com said: > On Tue, 2015-03-17 at 15:27 +, Wei Liu wrote: > > This looks most interesting as it implies we can easily pipe a console > > to it. > > BTW, rather than rawe consoles we should probably consider using the > channel extension: http://xenbits.xen.org/docs/unstable

app-tools toolchain naming (again)

2015-03-18 Thread Martin Lucina
Hi, having dealt with some more build systems (CMake for one), I have realised that the naming we agreed on for the app-tools toolchain back in January [1] is wrong, and should be fixed. Mea culpa. The current scheme names the tools as rumprun-{xen,bmk}-cc, ... This causes several problems: 1)

Re: app-tools toolchain naming (again)

2015-03-18 Thread Martin Lucina
mar...@lucina.net said: > Therefore, I propose to rename the tools yet again (hopefully for good) to > be correctly prefixed with the full autoconf target architecture, just like > any other cross-compilation toolchain. Antti pointed out to me on IRC that the above paragraph is a bit terse in spec

Re: app-tools toolchain naming (again)

2015-03-18 Thread Martin Lucina
po...@iki.fi said: > "obsolete" is a dangerous term when it comes to autoconf. > decade-old configure scripts are not rare in the real world. Personally I think the current -configure wrapper should go away and users should just cross compile using whatever method works best for their software. I

Deprecating / replacing the cc.configure "stub compiler"

2015-03-18 Thread Martin Lucina
Background: We currently have two different compilers in app-tools, the normal compiler and a "stub" compiler used by the -configure wrapper. The reason for the existence of the stub compiler is that a lot of configure tests use a full compile and link to determine if X compiles, symbol Y is prese

Specifying parameters to the toolchain (components to link, target board)

2015-03-18 Thread Martin Lucina
A couple more things have come to mind from the discussion so far which I think are worth bringing up: 1) We don't currently have a way of modifying the link list used for linking a Unikernel image, i.e. saying which Rump Kernel components are wanted or not. So, for example, I had to add -lrumpnet

Re: Upstream QEMU based stubdom and rump kernel

2015-03-18 Thread Martin Lucina
a...@recoil.org said: > > Point 2. will further require implementing support in the Rump Kernel, > > either for a shim which would proxy AF_UNIX / AF_INET transparently using > > vchan, or possibly later implementing a separate socket family (AF_VCHAN / > > AF_HYPER?). Once that is done you should

Installing the app-tools + rumprun toolchain

2015-03-18 Thread Martin Lucina
Last new thread for today, honest :-) I'd like to be able to install the app-tools and rumprun toolchain on the host system, i.e. provide a "make install". Again, this is about making things work as people would expect them to. AFAICT from some of the recent discussion on IRC about getting Xen OSS

Re: Deprecating / replacing the cc.configure "stub compiler"

2015-03-18 Thread Martin Lucina
po...@iki.fi said: > One string we could pull from in the future is the fact that we're > crosscompiling, and the build system doesn't know how to run the > resulting binary. So we could pseudo-link the target, and then have > rumprun actually link it (and cache the result). That approach > shoul

Re: Deprecating / replacing the cc.configure "stub compiler"

2015-03-19 Thread Martin Lucina
po...@iki.fi said: > On 18/03/15 20:11, Antti Kantee wrote: > >I think we should follow the adage "first make it right, > >then make it fast". If using a separate compiler for configure is not > >right, and we can't figure out any other solution without obvious > >drawbacks, I'd just as well have

Re: Upstream QEMU based stubdom and rump kernel

2015-03-19 Thread Martin Lucina
po...@iki.fi said: > Where do you propose to inject that faking out (and what does it > even mean)? Someone at Berkeley decided that socket drivers should > be globally enumerated, and PF_UNIX leads to exactly one handler. > Just hacking hooks as local patches into the PF_UNIX driver is > against

[DRAFT] Choosing an architecture tuple for Rumprun

2015-03-30 Thread Martin Lucina
Hi All, Regarding the recent threads on app-tools naming, et al. I'd like to get some advice from knowledgeable toolchain people before making a final choice. I have written the following draft to send to the GCC mailing list as that appears to be where most toolchain people hang out. Antti, Just

Re: [DRAFT] Choosing an architecture tuple for Rumprun

2015-03-30 Thread Martin Lucina
mar...@lucina.net said: > Antti, Justin, anyone else interested, can you please read through this and > let me know if its clear and if I have missed anything before I send it on > to the GCC folks? Thank you Antti, Justin and others on IRC for your comments and input. I've incorporated your comme

Re: [DRAFT] Choosing an architecture tuple for Rumprun

2015-03-30 Thread Martin Lucina
FYI, you can find the gcc thread in the archives here: https://gcc.gnu.org/ml/gcc/2015-03/msg00350.html Martin

Re: packaging system for rumprun

2015-04-09 Thread Martin Lucina
On Monday, 23.03.2015 at 19:48, Antti Kantee wrote: > Hi, > > This post is very handwavy just to put the subject on the radar. If > you're not interested in handwaving, better not read further ;) Handwavy reply coming up :-) > Much like a regular packaging system, the rumprun packaging system >

Re: configuration

2015-04-09 Thread Martin Lucina
On Wednesday, 01.04.2015 at 10:26, Justin Cormack wrote: > 3. Adding a file system to the image. I was looking that this before, > with my lightweight libuntar (25k) to untar an in memory image to the > root file system. The nice way to do this is to allow the user to > append the archive to the bi

Re: configuration

2015-04-09 Thread Martin Lucina
On Thursday, 09.04.2015 at 13:28, Justin Cormack wrote: > On 9 April 2015 at 13:26, Martin Lucina wrote: > > > >> > Regarding appending the fs to the resulting binary, you'd need to do more > >> > than just that. Haven't checked, but afaik eg. the Xen lo

Re: packaging system for rumprun

2015-04-09 Thread Martin Lucina
On Thursday, 09.04.2015 at 12:18, Antti Kantee wrote: > Can't the components just be built like they normally are: as > libraries? I don't see them being different from the rump kernel, > which is just libraries too. Yes, but the packaging system needs to know about them, in the sense that "nginx

Re: internal architecture of the unified rumprun repository

2015-04-09 Thread Martin Lucina
On Thursday, 09.04.2015 at 07:05, Antti Kantee wrote: > On 09/04/15 05:55, Hajime Tazaki wrote: > > > >Hi Antti, > > > >I drew a figure to try to understand the picture. > > > >https://www.dropbox.com/s/8ccu3dqq5vwodts/rumpkernel.gif?dl=0 > > Very helpful, thank you. > > #0 does not have to be "h

Re: configuration

2015-04-09 Thread Martin Lucina
On Thursday, 09.04.2015 at 13:12, Justin Cormack wrote: > On 9 April 2015 at 13:01, Martin Lucina wrote: > > On Wednesday, 01.04.2015 at 10:26, Justin Cormack wrote: > >> 3. Adding a file system to the image. I was looking that this before, > >> with my lightweight li

Re: configuration

2015-04-09 Thread Martin Lucina
On Wednesday, 01.04.2015 at 11:37, Antti Kantee wrote: > None of those involve the mechanism for *passing the configuration > from the tool to the guest*. That's an implementation detail. Yes, > the switches you list are missing (or at least some of them, didn't > check all), and we need to figur

Re: internal architecture of the unified rumprun repository

2015-04-09 Thread Martin Lucina
On Sunday, 05.04.2015 at 23:38, Antti Kantee wrote: > If you think the names suck, invent just better ones, and we can > change them without screwing users because they're exposed only > internally(!). The names are fine, however the internal prefix of "bmk" in headers and symbols is sucky. Will t

Re: packaging system for rumprun

2015-04-09 Thread Martin Lucina
On Tuesday, 24.03.2015 at 09:52, Antti Kantee wrote: > Let's say you want to create an nginx package which runs on Xen. I > think nginx needs some crosscompilation patches which are integrated > into the nginx available from: https://github.com/mato/rump-php > (logically ;) These are now split in

Re: Got some rump kernel web servers going but they are returning socket errors

2015-04-19 Thread Martin Lucina
andrew.stu...@supercoders.com.au said: > So after much fiddling I managed to get some C applications (web servers) > compiling as rump kernels, specifically thttpd and mathopd Great! > When I use the siege benchmarking tool to hit the server harder, after maybe > a few hundred or thousand succe

Re: Got some rump kernel web servers going but they are returning socket errors

2015-04-20 Thread Martin Lucina
On Monday, 20.04.2015 at 19:58, Andrew Stuart wrote: > I think I have compiled the correct version but not absolutely sure. It > would be helpful if there was some sort of “version” command on rumprun like > rump run -v that I could use to confirm I have successfully compiled the > version that

Re: Got some rump kernel web servers going but they are returning socket errors

2015-04-20 Thread Martin Lucina
On Tuesday, 21.04.2015 at 07:25, Andrew Stuart wrote: > >>Apparently the console prints went missing after one change. Try again and > >>see if you get more hints. > > I’ve tried again - no luck - nginx wouldn’t start - see errors below. > > Is there anyone else on the list willing to also try

Re: Got some rump kernel web servers going but they are returning socket errors

2015-04-20 Thread Martin Lucina
On Monday, 20.04.2015 at 22:43, Antti Kantee wrote: > >That'll be sigaction now using STUB_ERRNO() and returning -1, presumably > >due to Antti trying to find a fix for the times() problem and fixing the > >old STUB() in the process. > > > >This is a problem on our side, give us a bit to fix it. >

Re: Got some rump kernel web servers going but they are returning socket errors

2015-04-24 Thread Martin Lucina
On Friday, 24.04.2015 at 08:00, Antti Kantee wrote: > On 24/04/15 07:44, Andrew Stuart wrote: > >>>Before we speculate more, can you first test it again and see if the > >>>numbers got some X better? > > > > > >Start command is: > > > >rumprun xen -di -M 1024 \ > > -n inet,static,192.168.1.33/

Re: Got some rump kernel web servers going but they are returning socket errors

2015-04-24 Thread Martin Lucina
On Friday, 24.04.2015 at 08:14, Antti Kantee wrote: > >For comparison, I've just tried mathopd, and it does not suffer from the > >same problem: > > > >[snip] > > > >Transaction rate: 6676.46 trans/sec > >Transaction rate: 6589.34 trans/sec > >Transaction rate: 6657.97 trans

mathopd benchmarks - rumprun-xen vs. Linux vs. NetBSD - fight! socket errors

2015-04-24 Thread Martin Lucina
Hi, following up on the "web servers" thread with a better subject so that other people can see how we're doing with rump kernel powered web server performance on Xen, I present some totally unscientific benchmarks: The players: 1) rumprun-xen master, 128M RAM. 2) NetBSD 7.99.10 (XEN3_DOMU.2015

Re: on science (was: Re: mathopd benchmarks - rumprun-xen vs. Linux vs. NetBSD - fight! socket errors)

2015-04-24 Thread Martin Lucina
On Friday, 24.04.2015 at 10:02, Antti Kantee wrote: > On 24/04/15 09:23, Martin Lucina wrote: > >Hi, > > > >following up on the "web servers" thread with a better subject so that > >other people can see how we're doing with rump kernel powered web se

  1   2   3   >