[osv-dev] Trying to explore using OSv

2020-09-03 Thread Tim Hawes
I am trying to explore OSv and Unikernels in general. I have a Capstanfile 
defined in this link to a public, and the output when I build and run the 
app using capstan  gist 
https://gist.github.com/timotheosh/a98884e4d13fca669110df18cde054e2

Other useful details include that I am doing this from Pop!_OS 20.04 (an 
Ubuntu 20.04.1 derivative), I compiled the latest version of capstan using 
golang v1.15, and I am trying to run the osv image on kvm/qemu.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/e9a6ad1d-6e5f-4d0f-a298-7626e611524cn%40googlegroups.com.


[osv-dev] [PATCH v2 3/3] scripts/build: don't exit when exporting files

2020-09-03 Thread Fotis Xenakis
This allows exporting an image's files while doing a complete build on
the same invocation (these should anyway be orthogonal).

Signed-off-by: Fotis Xenakis 
---
 scripts/build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/scripts/build b/scripts/build
index 24e930d3..fed78fd3 100755
--- a/scripts/build
+++ b/scripts/build
@@ -271,7 +271,6 @@ fi
 if [ "$export" != "none" ]; then
export_dir=${vars[export_dir]-$SRC/build/export}
"$SRC"/scripts/export_manifest.py -e "$export_dir" -m usr.manifest -D 
libgcc_s_dir="$libgcc_s_dir"
-   exit 0
 fi
 
 if [[ ${vars[create_disk]} == "true" ]]; then
-- 
2.28.0

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/AM0PR03MB62929812DFA2CDBE61B46A77A62C0%40AM0PR03MB6292.eurprd03.prod.outlook.com.


[osv-dev] [PATCH v2 2/3] loader: add support for booting from virtio-fs

2020-09-03 Thread Fotis Xenakis
This makes the necessary and pretty straight-forward additions to the
laoder to support using virtio-fs as the root filesystem. It also makes
minimal changes to scripts/build to add support there as well. Note
that, to obtain a directory with contents specified by the manifest
files, usable as the virtio-fs host directory, one can use the existing
'export' and 'export_dir' (previously undocumented) options to
scripts/build.

Ref https://github.com/cloudius-systems/osv/issues/1062.

Signed-off-by: Fotis Xenakis 
---
 fs/vfs/main.cc | 24 
 loader.cc  | 10 +-
 scripts/build  | 22 +++---
 3 files changed, 48 insertions(+), 8 deletions(-)

diff --git a/fs/vfs/main.cc b/fs/vfs/main.cc
index 957333c2..f26e2275 100644
--- a/fs/vfs/main.cc
+++ b/fs/vfs/main.cc
@@ -2424,6 +2424,30 @@ extern "C" int mount_zfs_rootfs(bool pivot_root, bool 
extra_zfs_pools)
 return 0;
 }
 
+extern "C" int mount_virtiofs_rootfs(bool pivot_root)
+{
+constexpr char* mp = "/virtiofs";
+
+if (mkdir(mp, 0755) < 0) {
+int ret = errno;
+kprintf("failed to create %s, error = %s\n", mp, strerror(errno));
+return ret;
+}
+
+int ret = sys_mount("/dev/virtiofs0", mp, "virtiofs", MNT_RDONLY, nullptr);
+if (ret) {
+kprintf("failed to mount %s, error = %s\n", mp, strerror(ret));
+rmdir(mp);
+return ret;
+}
+
+if (pivot_root) {
+pivot_rootfs(mp);
+}
+
+return 0;
+}
+
 extern "C" void unmount_rootfs(void)
 {
 int ret;
diff --git a/loader.cc b/loader.cc
index 19b47e0e..9656cf3b 100644
--- a/loader.cc
+++ b/loader.cc
@@ -91,6 +91,7 @@ extern "C" {
 int mount_zfs_rootfs(bool, bool);
 int mount_rofs_rootfs(bool);
 void rofs_disable_cache();
+int mount_virtiofs_rootfs(bool);
 }
 
 void premain()
@@ -165,7 +166,7 @@ static void usage()
 std::cout << "  --leakstart leak detector after boot\n";
 std::cout << "  --nomount don't mount the root file system\n";
 std::cout << "  --nopivot do not pivot the root from bootfs to 
the root fs\n";
-std::cout << "  --rootfs=arg  root filesystem to use (zfs, rofs or 
ramfs)\n";
+std::cout << "  --rootfs=arg  root filesystem to use (zfs, rofs, 
ramfs or virtiofs)\n";
 std::cout << "  --assign-net  assign virtio network to the 
application\n";
 std::cout << "  --maxnic=arg  maximum NIC number\n";
 std::cout << "  --norandomdon't initialize any random 
device\n";
@@ -435,6 +436,13 @@ void* do_main_thread(void *_main_args)
 // TODO: Avoid the hack of using pivot_rootfs() just for mounting
 // the fstab entries.
 pivot_rootfs("/");
+} else if (opt_rootfs.compare("virtiofs") == 0) {
+auto error = mount_virtiofs_rootfs(opt_pivot);
+if (error) {
+debug("Could not mount virtiofs root filesystem.\n");
+}
+
+boot_time.event("Virtio-fs mounted");
 } else {
 // Fallback to original behavior for compatibility: try rofs -> zfs
 if (mount_rofs_rootfs(opt_pivot) == 0) {
diff --git a/scripts/build b/scripts/build
index 8fa18071..24e930d3 100755
--- a/scripts/build
+++ b/scripts/build
@@ -24,8 +24,9 @@ usage() {
  --help|-h Print this help message
  arch=x64|aarch64  Specify the build architecture; default 
is x64
  mode=release|debugSpecify the build mode; default is 
release
- export=none|selected|all  If 'selected' or 'all' export the app 
files to build/export
- fs=zfs|rofs|ramfs Specify the filesystem of the image 
partition
+ export=none|selected|all  If 'selected' or 'all' export the app 
files to 
+ export_dir=  The directory to export the files to; 
default is build/export
+ fs=zfs|rofs|ramfs|virtiofsSpecify the filesystem of the image 
partition
  fs_size=N Specify the size of the image in bytes
  fs_size_mb=N  Specify the size of the image in MiB
  app_local_exec_tls_size=N Specify the size of app local TLS in 
bytes; the default is 64
@@ -182,12 +183,17 @@ manifest=bootfs.manifest.skel
 fs_type=${vars[fs]-zfs}
 usrskel_arg=
 case $fs_type in
-zfs);; # Nothing to change here. This is our default behavior
-rofs)  manifest=bootfs_empty.manifest.skel
+zfs)
+   ;; # Nothing to change here. This is our default behavior
+rofs|virtiofs)
+   # Both are read-only (in OSv) and require nothing extra on bootfs to 
work
+   manifest=bootfs_empty.manifest.skel
usrskel_arg="--usrskel usr_rofs.manifest.skel";;
-ramfs) manifest=$OUT/usr.manifest
+ramfs)
+   manifest=$OUT/usr.manifest
usrskel_arg="--usrskel usr_ramfs.manifest.skel";;
-*) echo "Unknown filesystem \"$fs_type\"" >&2
+*)
+   

[osv-dev] [PATCH v2 1/3] loader: add rootfs type command line option

2020-09-03 Thread Fotis Xenakis
Previously, the loader always attempted to:
1. Mount a rofs root filesystem.
2. Mount a zfs root filesystem, if the previous failed.
3. Do nothing (i.e. keep the ramfs mounted as bootfs) if the others
   failed.

This adds a new command line option to better control this behaviour,
specifying which filesystem should be attempted for mounting root,
falling back to the above if not specified. This option is set by
scripts/build (scripts/run.py would be the obvious choice for setting a
command line option, but it would require the user specifying the root
fs type a run time in addition to build time).

In addition, in a way this formalizes supporting ramfs as the root
filesystem. As noted in the code, this support consists of simply
mounting the fstab entries when ramfs is selected (this is in sync with
what happened previously, when mounting zfs failed). So, there is no
functionality added here, just documenting the option and making the
code more clear and explicit.

Signed-off-by: Fotis Xenakis 
---
 fs/vfs/main.cc |  2 +-
 loader.cc  | 60 +-
 scripts/build  |  3 +++
 3 files changed, 54 insertions(+), 11 deletions(-)

diff --git a/fs/vfs/main.cc b/fs/vfs/main.cc
index 6cee319e..957333c2 100644
--- a/fs/vfs/main.cc
+++ b/fs/vfs/main.cc
@@ -2330,7 +2330,7 @@ static void mount_fs(mntent *m)
 }
 
 extern std::vector opt_mount_fs;
-void pivot_rootfs(const char* path)
+extern "C" void pivot_rootfs(const char* path)
 {
 int ret = sys_pivot_root(path, "/");
 if (ret)
diff --git a/loader.cc b/loader.cc
index 9d9d3173..19b47e0e 100644
--- a/loader.cc
+++ b/loader.cc
@@ -86,6 +86,7 @@ void setup_tls(elf::init_table inittab)
 extern "C" {
 void premain();
 void vfs_init(void);
+void pivot_rootfs(const char*);
 void unmount_devfs();
 int mount_zfs_rootfs(bool, bool);
 int mount_rofs_rootfs(bool);
@@ -134,6 +135,7 @@ bool opt_power_off_on_abort = false;
 static bool opt_log_backtrace = false;
 static bool opt_mount = true;
 static bool opt_pivot = true;
+static std::string opt_rootfs;
 static bool opt_random = true;
 static bool opt_init = true;
 static std::string opt_console = "all";
@@ -161,8 +163,9 @@ static void usage()
 std::cout << "  --trace=arg   tracepoints to enable\n";
 std::cout << "  --trace-backtrace log backtraces in the tracepoint 
log\n";
 std::cout << "  --leakstart leak detector after boot\n";
-std::cout << "  --nomount don't mount the ZFS file system\n";
-std::cout << "  --nopivot do not pivot the root from bootfs to 
the ZFS\n";
+std::cout << "  --nomount don't mount the root file system\n";
+std::cout << "  --nopivot do not pivot the root from bootfs to 
the root fs\n";
+std::cout << "  --rootfs=arg  root filesystem to use (zfs, rofs or 
ramfs)\n";
 std::cout << "  --assign-net  assign virtio network to the 
application\n";
 std::cout << "  --maxnic=arg  maximum NIC number\n";
 std::cout << "  --norandomdon't initialize any random 
device\n";
@@ -274,6 +277,14 @@ static void parse_options(int loader_argc, char** 
loader_argv)
 debug("console=%s\n", opt_console);
 }
 
+if (options::option_value_exists(options_values, "rootfs")) {
+auto v = options::extract_option_values(options_values, "rootfs");
+if (v.size() > 1) {
+printf("Ignoring '--rootfs' options after the first.");
+}
+opt_rootfs = v.front();
+}
+
 if (options::option_value_exists(options_values, "mount-fs")) {
 auto mounts = options::extract_option_values(options_values, 
"mount-fs");
 for (auto m : mounts) {
@@ -397,23 +408,52 @@ void* do_main_thread(void *_main_args)
 if (opt_mount) {
 unmount_devfs();
 
-// Try to mount rofs
-if (mount_rofs_rootfs(opt_pivot) != 0) {
-// Failed -> try to mount zfs
+if (opt_rootfs.compare("rofs") == 0) {
+auto error = mount_rofs_rootfs(opt_pivot);
+if (error) {
+debug("Could not mount rofs root filesystem.\n");
+}
+
+if (opt_disable_rofs_cache) {
+debug("Disabling ROFS memory cache.\n");
+rofs_disable_cache();
+}
+boot_time.event("ROFS mounted");
+} else if (opt_rootfs.compare("zfs") == 0) {
 zfsdev::zfsdev_init();
 auto error = mount_zfs_rootfs(opt_pivot, opt_extra_zfs_pools);
 if (error) {
 debug("Could not mount zfs root filesystem.\n");
 }
-bsd_shrinker_init();
 
+bsd_shrinker_init();
 boot_time.event("ZFS mounted");
+} else if (opt_rootfs.compare("ramfs") == 0) {
+// NOTE: The ramfs is already mounted, we just need to mount fstab
+// entries. That's the only difference between 

[osv-dev] [PATCH v2 0/3] Support booting from virtio-fs

2020-09-03 Thread Fotis Xenakis
Changes since v1:
- Made the loader backwards compatible: if the --rootfs flag is not
  specified, it fals back to the previous behavior of trying to mount
  rofs, followed by zfs.

On the root filesystem selection front, the thought of generalizing the
current implementation to try out the other fs as well when --rootfs is
not specified has passed through my mind. This could be a basic
auto-discovery process and should be straight-forward as long as the
various mount_*_rootfs are side-effect free when they fail. Do you think
it's something worth pursuing?

Fotis Xenakis (3):
  loader: add rootfs type command line option
  loader: add support for booting from virtio-fs
  scripts/build: don't exit when exporting files

 fs/vfs/main.cc | 26 ++-
 loader.cc  | 68 ++
 scripts/build  | 26 +--
 3 files changed, 101 insertions(+), 19 deletions(-)

-- 
2.28.0

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/AM0PR03MB62927D3618586AE600B4FBE1A62C0%40AM0PR03MB6292.eurprd03.prod.outlook.com.