Hello Roman/Friends, Roman, I'm having the freedom to share this message with the list once I'm not an expert in gvisor Actually, the idea is not to integrate gvisor, but instead take the list of implemented syscalls and check if we implemented them in OSv or not But theoretically, gvisor is written in go and go can be interfaced with assembly/c/c++/... So, theoretically, we might be able to replace OSv kernel by gvisor but that would have other implications as far as I know eg: gvisor is a user space implementation of linux kernel which means we would still need to provide some linux ABI somehow Maybe someone from the list might be able to correct me if I'm saying some no-sense...
Kind Regards, Geraldo Netto Sapere Aude => Non dvcor, dvco http://exdev.sf.net/ On Tue, 13 Nov 2018 at 20:14, Roman Shaposhnik <[email protected]> wrote: > On Tue, Nov 13, 2018 at 11:10 AM Geraldo Netto <[email protected]> > wrote: > > > > Hi Waldek/Friends! > > > > As always, it's very interesting! > > If you allow me to suggest another thing is to review the syscalls that > are most used > > So, we implement them from time to time > > I think gvisor from google might give an interesting idea once if I > understood it correctly it's a linux kernel implementation in go > > and they probably don't support all syscalls but only the most used ones > > I volunteer myself to check this if you find interesting > > I am curious: what kind of integration with gvisor do you have in mind? > > Thanks, > Roman. > > > Do You/Nadav happen to have any suggestion? > > > > Also, I would suggest to review the wiki page, see if all information > are still valid > > and possibily include it in the code, once there are troubleshooting and > information that might be relevant when running OSv only in terminal > > > > As from my personal side, I'm still interested in working (just for fun) > in the following things: > > - finish tomcat9 upgrade ("a promise is a debt") > > - finish the python app build script (it's working but incomplete, I > might upload to github any time soon) > > - musl upgrade > > - modular syscall (check application code recursively to disable all > unused syscalls, honestly, i don't think I'll ever finish this task, but > once the idea is interesting enough, i'll keep listing...) > > > > > > Kind Regards, > > > > Geraldo Netto > > Sapere Aude => Non dvcor, dvco > > http://exdev.sf.net/ > > > > > > On Tue, 13 Nov 2018 at 19:54, Waldek Kozaczuk <[email protected]> > wrote: > >> > >> This is somewhat revised version of the roadmap I sent couple of months > ago. I think that the main premise of it which I wrote in the original > version still stands: > >> > >> "In my opinion, if OSv is to become more relevant than it is now, its > primary target should be a platform for running stateless and serverless > apps. Therefore it needs to become leaner (memory usage and kernel size), a > little more modular and boot faster while preserving its functionality in > terms of Linux/POSIX ABI (and possibly expanding where it makes sense). > That is not say that we should not improve support for stateful apps like > mysql or elasticsearch which would require optimizations to VFS and > possibly ZFS which I list below." > >> > >> OBJECTIVES > >> > >> Make OSv more modular > >> > >> ZFS and commands/runscript functionality should be moved as optional > modules in form of shared libraries > >> The key building block to achieve it should be an improved ROFS > intended as a default filesystem to load code from > >> > >> Optimize kernel size > >> > >> My experiments show that kernel (loader-stripped.elf) can be as small > as 5.1MB after ZFS and program options are compiled out (and there is still > room for improvements) > >> > >> Optimize boot time > >> > >> OSv can execute under 50ms on hyperkit and under 100ms on QEMU with > qboot > >> Smaller kernel should reduce boot time > >> > >> Optimize memory usage > >> > >> My recent investigation about memory allocation is OSv shows that it > should be possible and quite easy to reduce memory utilization in many areas > >> The aggressive but quite realistic goal should be an ability to run OSv > with simple C app using no more than 10MB of RAM > >> Smaller kernel should also reduce memory usage > >> > >> > >> MORE IMPORTANT > >> File systems > >> > >> ROFS > >> > >> [FASTER BOOT] Add compression - > https://github.com/cloudius-systems/osv/issues/978 > >> [LESS MEMORY] Avoid creating another copy when mmap-ing ROFS file - > https://github.com/cloudius-systems/osv/issues/979 > >> > >> ZFS > >> > >> [MODULARITY, SMALLER KERNEL, FASTER BOOT] move all ZFS code in kernel > into libzfs.so > >> > >> Make ZFS an optional library - > https://github.com/cloudius-systems/osv/issues/1009 > >> > >> RAMFS > >> > >> [SPEED, LESS MEMORY] Fix slow write/append of files on RAMFS - > https://github.com/cloudius-systems/osv/issues/884 > >> Eliminate BootFS > >> > >> 9pFS > >> > >> [ENHANCEMENTS] https://github.com/cloudius-systems/osv/issues/1008 > >> > >> > >> Optimize memory usage > >> > >> Wasted memory in early (pre-SMP-enabled) malloc (possibly save 6MB of > memory!!!) - https://github.com/cloudius-systems/osv/issues/270 > >> Make allocations < 16 bytes more space efficient - > https://github.com/cloudius-systems/osv/issues/1011 > >> Improve physical memory utilization by using memory below 2MB - > https://github.com/cloudius-systems/osv/issues/1012 > >> Make L1/L2 memory pool sizes self-configurable depending on physical > memory available - https://github.com/cloudius-systems/osv/issues/1013 > >> Consider more space-efficient allocation between 1024 and 4096 bytes - > https://github.com/cloudius-systems/osv/issues/1000 > >> > >> > >> Improve modularity > >> > >> Remove boost programs_options from kernel by rewriting > loader.cc::parse_options() to use getopt_long() and extract > commands.cc:parse_command_line(const std::string line, bool& ok) as a > optional library commands.so; see > https://github.com/cloudius-systems/osv/issues/1014 for details > >> > >> > >> Optimize kernel size > >> > >> Be more selective on symbols exported from the kernel - > https://github.com/cloudius-systems/osv/issues/97 > >> > >> Remove/hide symbols > >> > >> Investigate what else can be discarded using bloaty > >> > >> eliminate some debug strings in BSD code tree that should further > reduce .rodata section > >> remove/hide symbols (reduces ELF .dynsym (others?) section) - > https://github.com/cloudius-systems/osv/issues/97 > >> reduce size of other sections in loader.elf > >> > >> > >> Other > >> > >> Make more programs run without having to re-compile (on ubuntu already > pies) > >> > >> look at adding missing symbols to make coreutils (ls, find) programs > run (sent email about it) > >> > >> Stop using libraries and headers from external - > https://github.com/cloudius-systems/osv/issues/743 and related … > https://github.com/cloudius-systems/osv/issues/687 > >> Assertion failed: timestamp >= _last (timer-set.hh: expire) - > https://github.com/cloudius-systems/osv/issues/382 > >> > >> > >> COMPLETE BUT NOT COMMITTED PATCHES > >> > >> [SECURITY] Read-only REST API > >> > >> https://github.com/cloudius-systems/osv/issues/820 > >> > >> mprotect() syscall stack > >> > >> we could commit it as is but then it would syscall stack use more > memory (1 page per thread) until we fix this issue - > https://github.com/cloudius-systems/osv/issues/1000 > >> > >> full revival of multiboot > >> > >> there may be still controversial issues resolved before this patch is > ready to be merged > >> > >> > >> LESS IMPORTANT > >> > >> [ENHANCEMENTS] Support new hypervisors > >> > >> qboot > >> NEMU > >> Hyperkit - https://github.com/cloudius-systems/osv/issues/948 > >> > >> multiboot - https://github.com/cloudius-systems/osv/issues/981 > >> > >> File systems > >> > >> RAMFS > >> > >> [LESS MEMORY] Avoid creating another copy when mmap-ing RAMF - > https://github.com/cloudius-systems/osv/issues/979 > >> [BUG] readdir does not handle asynchronous entry removal properly - > https://github.com/cloudius-systems/osv/issues/68 > >> > >> ZFS > >> > >> [ENHANCEMENTS] create ZFS image on host instead of booting OSv - see > https://github.com/cloudius-systems/osv/issues/918 > >> > >> Make VFS locking more granular > >> > >> VFS: read()/write() lock the vnode for too long - > https://github.com/cloudius-systems/osv/issues/450 > >> > >> this commit addresses ZFS - > https://github.com/cloudius-systems/osv/commit/b5eadc37f12c4b97a52705830d2d9097498049c2 > >> > >> people reported that some FS apps did not scale even though commit > above addresses > >> this commit > https://github.com/cloudius-systems/osv/commit/697943f631960b3d55f66381581e1435b726a1a8 > reverted previous one because of > https://github.com/cloudius-systems/osv/issues/504 > >> > >> Async IO - https://github.com/cloudius-systems/osv/issues/656 > >> > >> [MONITORING] REST API enhancements > >> > >> Remote test execution - > https://github.com/cloudius-systems/osv/issues/445 > >> Page in/page out - https://github.com/cloudius-systems/osv/issues/465 > >> Block IO - https://github.com/cloudius-systems/osv/issues/466 > >> Network stat - https://github.com/cloudius-systems/osv/issues/467 > >> Load average - https://github.com/cloudius-systems/osv/issues/469 > >> > >> [SECURITY] OpenSSL upgrade > >> Do not use setimg.py in Makefile - > https://github.com/cloudius-systems/osv/issues/733 > >> > >> > >> IN PROGRESS > >> > >> Add support for Mono, C# - > https://github.com/cloudius-systems/osv/issues/34 > >> > >> added sample app > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "OSv Development" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > >> For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > Groups "OSv Development" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
