Thanks! Nice plan. I have few small comments below. On Tue, Nov 13, 2018 at 8:54 PM Waldek Kozaczuk <[email protected]> wrote:
> This is somewhat revised version of the roadmap I sent couple of months > ago. I think that the main premise of it which I wrote in the original > version still stands: > > "In my opinion, if OSv is to become more relevant than it is now, its > primary target should be a platform for running stateless and serverless > apps. Therefore it needs to become leaner (memory usage and kernel size), a > little more modular and boot faster while preserving its functionality in > terms of Linux/POSIX ABI (and possibly expanding where it makes sense). > That is not say that we should not improve support for stateful apps like > mysql or elasticsearch which would require optimizations to VFS and > possibly ZFS which I list below." > > *OBJECTIVES* > > - Make OSv more modular > - > - ZFS and commands/runscript functionality should be moved as > optional modules in form of shared libraries > > Because this adds complexity (to the build system, to the images, etc.) we should probably investigate the benefits (in image size, memory size, etc.) before actually doing this. Also, in the case of ZFS, I wonder if there's a benefit of doing ZFS as a shared object or just have a build option for enabling and disabling it in the main kernel. I.e., if someone uses scripts/build to build an image with ZFS, or capstan which always assumes ZFS, we can compile the kernel with ZFS. About the commands thing: Personally I think command-line parsing is so trivial that it would be sad to take it out just because we used bloated Boost features (Qi and Options). More on this below. > > - > - The key building block to achieve it should be an improved ROFS > intended as a default filesystem to load code from > - Optimize kernel size > - > - My experiments show that kernel (loader-stripped.elf) can be as > small as 5.1MB after ZFS and program options are compiled out (and > there is > still room for improvements) > - Optimize boot time > - > - OSv can execute under 50ms on hyperkit and under 100ms on QEMU > with qboot > - Smaller kernel should reduce boot time > - Optimize memory usage > - > - My recent investigation about memory allocation is OSv shows that > it should be possible and quite easy to reduce memory utilization in > many > areas > - The aggressive but quite realistic goal should be an ability to > run OSv with simple C app using *no more than 10MB of RAM* > - Smaller kernel should also reduce memory usage > > > MORE IMPORTANT > File systems > > - ROFS > - > - [FASTER BOOT] Add compression - > https://github.com/cloudius-systems/osv/issues/978 > - [LESS MEMORY] Avoid creating another copy when mmap-ing ROFS file > - https://github.com/cloudius-systems/osv/issues/979 > - ZFS > - > - [MODULARITY, SMALLER KERNEL, FASTER BOOT] move all ZFS code in > kernel into libzfs.so > - > - Make ZFS an optional library - > https://github.com/cloudius-systems/osv/issues/1009 > - RAMFS > - > - [SPEED, LESS MEMORY] Fix slow write/append of files on RAMFS - > https://github.com/cloudius-systems/osv/issues/884 > - Eliminate BootFS > > You want to eliminate this filesystem (why?) or some of the files we now put in it by default? > > - 9pFS > - > - [ENHANCEMENTS] https://github.com/cloudius-systems/osv/issues/1008 > > > Optimize memory usage > > - Wasted memory in early (pre-SMP-enabled) malloc (*possibly save 6MB > of memory!!!*) - https://github.com/cloudius-systems/osv/issues/270 > - Make allocations < 16 bytes more space efficient - > https://github.com/cloudius-systems/osv/issues/1011 > > This is a really low hanging fruit that should definitely be fixed :-) > > - > - Improve physical memory utilization by using memory below 2MB - > https://github.com/cloudius-systems/osv/issues/1012 > - Make L1/L2 memory pool sizes self-configurable depending on physical > memory available - https://github.com/cloudius-systems/osv/issues/1013 > - Consider more space-efficient allocation between 1024 and 4096 bytes > - https://github.com/cloudius-systems/osv/issues/1000 > > > Improve modularity > > - Remove boost programs_options from kernel by rewriting > loader.cc::parse_options() to use getopt_long() and extract > commands.cc:parse_command_line(const > std::string line, bool& ok) as a optional library commands.so; see > https://github.com/cloudius-systems/osv/issues/1014 for details > > I think moving the parsing - both the boost options and the boost Qi (used for parsing ";", quotes, etc. in the command line) to a separate shared object, indeed makes sense as an easy first step. It means OSv can load this library, parse the options, and then unload it. However, I'm worried we have a chicken-and-egg problem in the sense that we want to store this library in ZFS not in bootfs (otherwise we don't save any memory), and we only mount ZFS after we parse the kernel's options? I wonder if it wouldn't make sense to just rewrite the command line parsing using getopt_long() plus perhaps (if it saves code size...) a simple state machine for the Qi stuff, and keep it in the kernel after it becomes a tiny piece of code. But maybe it's not worth the effort... > > - > > > Optimize kernel size > > - Be more selective on symbols exported from the kernel - > https://github.com/cloudius-systems/osv/issues/97 > - > - Remove/hide symbols > - Investigate what else can be discarded using bloaty > - > - eliminate some debug strings in BSD code tree that should further > reduce .rodata section > > What "debug strings"? > > - > - remove/hide symbols (reduces ELF .dynsym (others?) section) - > https://github.com/cloudius-systems/osv/issues/9 > <https://github.com/cloudius-systems/osv/issues/97>7 > - reduce size of other sections in loader.elf > > > Other > > - Make more programs run without having to re-compile (on ubuntu > already pies) > - > - look at adding missing symbols to make coreutils (ls, find) > programs run (sent email about it) > - Stop using libraries and headers from external - > https://github.com/cloudius-systems/osv/issues/743 and related … > https://github.com/cloudius-systems/osv/issues/687 > - Assertion failed: timestamp >= _last (timer-set.hh: expire) - > https://github.com/cloudius-systems/osv/issues/382 > > I would really love for this 4 year old bug to go away, but unfortunately could never figure it out. I came across an interesting observation - that the jump is always exactly 9 * 2^32 ns, and only specific to HPET - but was never able to explain why it happens. > > - > > > COMPLETE BUT NOT COMMITTED PATCHES > > - [SECURITY] Read-only REST API > - > - https://github.com/cloudius-systems/osv/issues/820 > - mprotect() syscall stack > - > - we could commit it as is but then it would syscall stack use more > memory (1 page per thread) until we fix this issue - > https://github.com/cloudius-systems/osv/issues/1000 > - full revival of multiboot > - > - there may be still controversial issues resolved before this > patch is ready to be merged > > > LESS IMPORTANT > > - [ENHANCEMENTS] Support new hypervisors > - > - qboot > - NEMU > - Hyperkit - https://github.com/cloudius-systems/osv/issues/948 > - > - multiboot - https://github.com/cloudius-systems/osv/issues/981 > - File systems > - > - RAMFS > - > - [LESS MEMORY] Avoid creating another copy when mmap-ing RAMF - > https://github.com/cloudius-systems/osv/issues/979 > - [BUG] readdir does not handle asynchronous entry removal > properly - https://github.com/cloudius-systems/osv/issues/68 > - ZFS > - > - [ENHANCEMENTS] create ZFS image on host instead of booting OSv > - see https://github.com/cloudius-systems/osv/issues/918 > - Make VFS locking more granular > - > - VFS: read()/write() lock the vnode for too long - > https://github.com/cloudius-systems/osv/issues/450 > - > - this commit addresses ZFS - > > https://github.com/cloudius-systems/osv/commit/b5eadc37f12c4b97a52705830d2d9097498049c2 > - > - people reported that some FS apps did not scale even > though commit above addresses > - this commit > > https://github.com/cloudius-systems/osv/commit/697943f631960b3d55f66381581e1435b726a1a8 > reverted > previous one because of > https://github.com/cloudius-systems/osv/issues/504 > - Async IO - > https://github.com/cloudius-systems/osv/issues/656 > - [MONITORING] REST API enhancements > - > - Remote test execution - > https://github.com/cloudius-systems/osv/issues/445 > - Page in/page out - > https://github.com/cloudius-systems/osv/issues/465 > - Block IO - https://github.com/cloudius-systems/osv/issues/466 > - Network stat - https://github.com/cloudius-systems/osv/issues/467 > - Load average - https://github.com/cloudius-systems/osv/issues/469 > - [SECURITY] OpenSSL upgrade > - Do not use setimg.py in Makefile - > https://github.com/cloudius-systems/osv/issues/733 > > > IN PROGRESS > > - Add support for Mono, C# - > https://github.com/cloudius-systems/osv/issues/34 > - > - added sample app > > -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
