Thanks for spending time on modernizing this README. I know you already committed it, but still, some comments below.
On Mon, Aug 19, 2019 at 6:57 AM Commit Bot <[email protected]> wrote: > From: Waldemar Kozaczuk <[email protected]> > Committer: Waldemar Kozaczuk <[email protected]> > Branch: master > > docs: Updated main README to make it better reflect current state of OSv > > Signed-off-by: Waldemar Kozaczuk <[email protected]> > > --- > diff --git a/README.md b/README.md > --- a/README.md > +++ b/README.md > @@ -1,115 +1,103 @@ > # OSv > > -OSv is a new open-source operating system for virtual-machines. > -OSv was designed from the ground up to execute a single application on top > -of a hypervisor, resulting in superior performance and effortless > management > -when compared to traditional operating systems which were designed for > -a vast range of physical machines. > I wonder why you didn't like this paragraph. It spells out some things - like the intention to run a single application, and running on a hypervisor not a physical machine - which are less clear in your new version, I think. That being said, I'm not opposed to your new text. I'll just need to get used to it :-) - > -OSv has new APIs for new applications, but also runs unmodified Linux > -applications (most of Linux's ABI is supported) and in particular can run > -an unmodified JVM, and applications built on top of one. > - > -For more information about OSv, see http://osv.io/ and > -https://github.com/cloudius-systems/osv/wiki > - > -## Documentation > - > -* [Debugging > OSv](https://github.com/cloudius-systems/osv/wiki/Debugging-OSv) > -* [Trace > Analysis]( > https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py > ) > - > -## Building > +OSv is an open-source versatile modular **unikernel** designed to run > In what sense is OSv "modular"? In some senses it is even less modular than Linux. > **unmodified > +Linux applications** securely on micro-VMs in the cloud. How are "micro VMs" different from regular VMs? And it's not just "in the cloud", it's any virtual machines (e.g., the arm use-cases were about using OSv in VMs running in cars or routers). > Built from > +the ground up for effortless deployment and management of microservices > +and serverless apps, with superior performance. > + > +OSv has new APIs for new applications, I think these APIs are more-or-less abandoned, so maybe we shouldn't even bother to mention them :-( but also runs unmodified x86-64 > Linux > +binaries **as is**, which effectively makes it a **Linux binary > compatible > unikernel** > +(for more details about Linux ABI compatibility please read > +[this > doc]( > https://github.com/cloudius-systems/osv/wiki/OSv-Linux-ABI-Compatibility) > ). > +In particular OSv can run many managed language runtimes including > unmodified > +[**JVM**]( > https://github.com/cloudius-systems/osv-apps/tree/master/java-example), > +**Python** > [**2**](https://github.com/cloudius-systems/osv-apps/tree/master/python2x) > > and > +[**3**](https://github.com/cloudius-systems/osv-apps/tree/master/python3x > ), > +[**Node.JS**]( > https://github.com/cloudius-systems/osv-apps/tree/master/node-from-host), > +[**Ruby**]( > https://github.com/cloudius-systems/osv-apps/tree/master/ruby-example), > **Erlang**, > and applications built on top of one. > +It can also run applications written in languages compiling directly to > native machine code like > +**C**, **C++**, > [**Golang**]( > https://github.com/cloudius-systems/osv-apps/tree/master/golang-httpserver > ) > +and > [**Rust**]( > https://github.com/cloudius-systems/osv-apps/tree/master/rust-httpserver) > > as well as native images produced > +by > [**GraalVM**]( > https://github.com/cloudius-systems/osv-apps/tree/master/graalvm-example). > + > +OSv can boot as fast as **~5 ms** on Firecracker using as low as 15 MiB > of > memory. > +OSv can run on many hypervisors including QEMU/KVM, > +[Firecracker]( > https://github.com/cloudius-systems/osv/wiki/Running-OSv-on-Firecracker), > +Xen, > [VMWare]( > https://github.com/cloudius-systems/osv/wiki/Running-OSv-on-VMware-ESXi), > +[VirtualBox]( > https://github.com/cloudius-systems/osv/wiki/Running-OSv-on-VirtualBox) > and > +Hyperkit as well as open clouds like AWS EC2, GCE and OpenStack. > + > +For more information about OSv, see [the main wiki > page](https://github.com/cloudius-systems/osv/wiki) > +and http://osv.io/. > + > +## Building and Running Apps on OSv > + > +In order to run an application on OSv, one needs to build an image by > fusing OSv kernel and > +the application files together. This, in high level can be achieved in > two > ways: > +- by using the script > [build](https://github.com/cloudius-systems/osv/blob/master/scripts/build) > + that builds the kernel from source and fuses it with application files > +- by using the [capstan tool](https://github.com/cloudius-systems/capstan) > > that uses *pre-built kernel* > + and combines it with application files to produce final image > + > +If your intention is to try to run your app on OSv with least effort, > you > should pursue the *capstan* > +route. For introduction please read this [crash > course]( > https://github.com/cloudius-systems/osv/wiki/Build-and-run-apps-on-OSv-using-Capstan > ) > +and for more details about capstan read [this > documentation](https://github.com/cloudius-systems/capstan#documentation). > + > +If you are comfortable with make and GCC toolchain and want to try the > latest OSv code, then you should > +read remaining part of this page to guide you how to setup your > development environment and build OSv kernel > +and application images. > + > +## Setting up development environment > > OSv can only be built on a 64-bit x86 Linux distribution. Please note > that > this means the "x86_64" or "amd64" version, not the 32-bit "i386" > version. > > -If you wish, you can run the script 'scripts/setup.py' as root to > install > all dependencies. > -Otherwise, you may follow the manual instructions below. > - > -First, install prerequisite packages: > - > -**Fedora** > +In order to build OSv kernel you need a physical or virtual machine with > Linux distribution on it and GCC toolchain and > +all necessary packages and libraries OSv build process depends on. The > easiest way to set it up is to use > +[Docker > files]( > https://github.com/cloudius-systems/osv/tree/master/docker#docker-osv-builder) > > that OSv comes with. > I'm not sure docker is the "easiest" solution. I think that if you have one of the distributions supported by scripts/setup.py, it is much easier. Isn't it? +You can use them to build your own Docker image and then start it in > order > to build OSv kernel inside of it. > > -``` > -scripts/setup.py > +Otherwise, you can manually clone OSv repo and use > [setup.py]( > https://github.com/cloudius-systems/osv/blob/master/scripts/setup.py) > Why did you change this text? script/setup.py is a file in the source repository. The user is supposed to just run it. Not to download something from the link you gave here. > +to install GCC Why did you single out "gcc"? Most users will already have gcc installed, it is the rest of the things which are likely to be missing. > and all required packages, as long as it supports your > Linux distribution and you have both git and python 2.7 installed on your > machine: > Why do you need python2.7 installed? Does scripts/setup.py not work with other versions of python? +```bash > Let's just just ```, not ```bash - people who read this as text won't know what this "bash" word means here. +git clone https://github.com/cloudius-systems/osv.git > +cd osv && git submodule update --init --recursive > +./scripts/setup.py > ``` > Isn't this git clone, submodule update, etc., thing needed *also* for the docker version? I think it's confusing, it sounds like these instructions are specific to setup.py. > > -**Debian stable(wheezy)** > -Debian stable(wheezy) requires to compile gcc, gdb and qemu. > -And also need to configure bridge manually. > +The ```setup.py``` recognizes and installs packages for number of Linux > Should be just one back-quote, not three (three are used to start a block) distributions including Fedora, Ubuntu, > [Debian]( > https://github.com/cloudius-systems/osv/wiki/Building-OSv-on-Debian-stable), > > LinuxMint and RedHat ones (Scientific Linux, NauLinux, CentOS Linux, Red > Hat Enterprise Linux, Oracle Linux). Please note that only Ubuntu and > Fedora support is actively maintained and tested so your milage with > other > distributions may vary. > This text is good, but it's the second time that setup.py is described, I think some more editing of this text is needed. > -More details are available on wiki page: > -[Building OSv on Debian stable][] > +## Building OSv kernel and creating images > > -[Building OSv on Debian stable]: > https://github.com/cloudius-systems/osv/wiki/Building-OSv-on-Debian-stable > +Building OSv is as easy as using the shell script > [build](https://github.com/cloudius-systems/osv/blob/master/scripts/build) > +that orchestrates the build process by delegating to the main > [makefile](https://github.com/cloudius-systems/osv/blob/master/Makefile) > +to build the kernel and by using number of Python scripts like > [module.py]( > https://github.com/cloudius-systems/osv/blob/master/scripts/module.py) > to build application and *fuse* it together with the kernel > +into a final image placed at ./build/release/usr.img > (or ./build/$(arch)/usr.img in general). Please note that *building app* > does > +not necessarily mean building from source as in many cases the app files > would be simply located on and taken from the Linux build machine > +(see > [manifest_from_host.sh]( > https://github.com/cloudius-systems/osv/blob/master/scripts/manifest_from_host.sh) > > for details). > > -**Debian testing(jessie)** > -``` > -apt-get install build-essential libboost-all-dev genromfs autoconf > libtool > openjdk-7-jdk ant qemu-utils maven libmaven-shade-plugin-java python-dpkt > tcpdump gdb qemu-system-x86 gawk gnutls-bin openssl python-requests > lib32stdc++-4.9-dev p11-kit > -``` > +The build script can be used like so per the examples below: > +```bash > +# Create default image that comes with command line and REST API server > +./scripts/build > > -**Arch Linux** > -``` > -pacman -S base-devel git python apache-ant maven qemu gdb boost yaml-cpp > unzip openssl-1.0 > -``` > +# Create image with native-example app > +./scripts/build -j4 fs=rofs image=native-example > > -Apply the following patch to make it work with openssl-1.0 > -``` > -diff --git a/modules/lua/Makefile b/modules/lua/Makefile > -index 9676f349..ddb6a075 100644 > ---- a/modules/lua/Makefile > -+++ b/modules/lua/Makefile > -@@ -123,7 +123,7 @@ $(CDIR)/ssl.lua: $(LUA_ROCKS_BIN) > - > - # Workaround because LuaRocks ignores /lib64 > - ifneq ("$(wildcard /usr/lib64/libssl.so*)", "") > -- out/bin/luarocks install LuaSec 0.5 OPENSSL_LIBDIR=/usr/lib64 > -+ out/bin/luarocks install LuaSec 0.5 > OPENSSL_LIBDIR=/usr/lib/openssl-1.0 OPENSSL_INCDIR=/usr/include/openssl-1.0 > - else > - out/bin/luarocks install LuaSec 0.5 > - endif > -``` > - > -Before start building OSv, you'll need to add your account to kvm group. > -``` > -usermod -aG kvm <user name> > -``` > - > -**Ubuntu**: > +# Create image with spring boot app with Java 10 JRE > +./scripts/build JAVA_VERSION=10 > image=openjdk-zulu-9-and-above,spring-boot-example > > -``` > -scripts/setup.py > -``` > - > -You may use [Oracle JDK][] if you don't want to pull too many > -dependencies for ``openjdk-7-jdk`` > + # Create image with 'ls' executable taken from the host > +./scripts/manifest_from_host.sh -w ls && ./script/build --append-manifest > > -[Oracle JDK]: https://launchpad.net/~webupd8team/+archive/java > +# Create test image and run all tests in it > +./script/build check > > -To ensure functional C++11 support, Gcc 4.8 or above is required, as > this > was > -the first version to fully comply with the C++11 standard. > - > -Make sure all git submodules are up-to-date: > - > -``` > -git submodule update --init --recursive > +# Clean the build tree > +./script/build clean > ``` > > -Finally, build everything at once: > - > -``` > -make -j$(nproc) > -``` > - > -to build only the OSv kernel, or more usefully, > - > -``` > -scripts/build -j$(nproc) > -``` > - > -to build an image of the OSv kernel and the default application. > - > -Command nproc (embedded in bash/core-utils) will calculte the number of > jobs/threads for make and scripts/build automatically. > +Command nproc will calculate the number of jobs/threads for make and > scripts/build automatically. > Alternatively, the environment variable MAKEFLAGS can be exported as > follows: > > ``` > @@ -118,45 +106,46 @@ export MAKEFLAGS=-j$(nproc) > > In that case, make and scripts/build do not need the parameter -j. > > -scripts/build creates the image ```build/last/usr.img``` in qcow2 format. > +For details on how to use the build script, please run > ```./scripts/build > --help```. > + > +The ```.scripts/build``` creates the image ```build/last/usr.img``` in > qcow2 format. > To convert this image to other formats, use the ```scripts/convert``` > -tool, which can create an image in the vmdk, vdi, raw, or qcow2-old > formats > -(qcow2-old is an older qcow2 format, compatible with older versions of > QEMU). > +tool, which can create an image in the vmdk, vdi or raw formats. > For example: > > ``` > scripts/convert raw > ``` > > -By default make will use the static libraries and headers of gcc in > external submodule. To change this pass `host` via *_env variables: > - > -``` > -make build_env=host > -``` > - > -This will use static libraries and headers in the system instead (make > sure they are installed before run `make`), > -if you only want to use C++ static libraries in the system, just set > `cxx_lib_env` to `host`: > - > -``` > -make cxx_lib_env=host > +By default OSv builds kernel for x86_64 architecture but it is also > possible to build one for ARM by adding **arch** parameter like so: > +```bash > +./scripts/build arch=aarch64 > ``` > +Please note that even though the **aarch64** version of OSv kernel > should > build fine, most likely it will **not** run as the ARM part of OSv has > not > been well maintained and tested due to the lack of volunteers. > > ## Running OSv > > -``` > +Running an OSv image, built by ```scripts/build```, is as easy as: > +```bash > ./scripts/run.py > ``` > > -By default, this runs OSv under KVM, with 4 VCPUs and 2GB of memory, > -and runs the default management application containing a shell, a > -REST API server and a browser base dashboard (at port 8000). > +By default, the ```run.py``` runs OSv under KVM, with 4 VCPUs and 2GB of > AFAIK, should be one quote, not three. memory. You can control these and tens of other ones by passing relevant > parameters to the ```run.py```. For details on how to use the script > please > run ```./scripts/run.py --help```. > + > +The ```run.py``` can run OSv image on QEMU/KVM, Xen and VMware. If > running > under KVM you can terminate by hitting Ctrl+A X. > > -If running under KVM you can terminate by hitting Ctrl+A X. > +Alternatively you can use ```./scripts/firecracker.py``` to run OSv on > [Firecracker](https://firecracker-microvm.github.io/). This script > automatically downloads firecracker and accepts number of parameters like > number ot VCPUs, memory named exactly like ```run.py``` does. > > +Please note that in order to run OSv with best performance on Linux > under > QEMU or Firecracker you need KVM enabled (this is only possible on > *physical* Linux machines, EC2 bare metal instances or VMs that support > nested virtualization with KVM on). The easiest way to verify KVM is > enabled is to check if ```/dev/kvm``` is present and your user can read > from and write to it. Adding your user to the kvm group may be necessary > like so: > +```bash > +usermod -aG kvm <user name> > +``` > + > +For more information about building and running JVM, Node.JS, Python and > other managed runtimes as well as Rust, Golang or C/C++ apps on OSv, > please > read this [wiki > page]( > https://github.com/cloudius-systems/osv/wiki#running-your-application-on-osv). > > For more information about various example apps you can build and run on > OSv, please read [the osv-apps repo > README](https://github.com/cloudius-systems/osv-apps#osv-applications). > > -## External Networking > +### Networking > > -To start osv with external networking: > +By default the ```run.py``` starts OSv with [user > networking/SLIRP]( > https://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29) > > on. To start OSv with more performant external networking: > > ``` > sudo ./scripts/run.py -n -v > @@ -165,7 +154,7 @@ sudo ./scripts/run.py -n -v > The -v is for kvm's vhost that provides better performance > and its setup requires a tap and thus we use sudo. > > -By default OSv spawns a dhcpd that auto config the virtual nics. > +By default OSv spawns a dhcpd that automatically configures the virtual > nics. > Static config can be done within OSv, configure networking like so: > > ``` > @@ -179,14 +168,8 @@ Test networking: > test invoke TCPExternalCommunication > ``` > > -## Running Java or C applications that already reside within the image: > +## Debugging, Monitoring, Profiling OSv > > -``` > -# Building and running a simple java application example > -$ scripts/build image=java-example > -$ scripts/run.py -e "java.so -cp /java-example Hello" > - > -# Running an ifconfig by explicit execution of ifconfig.so (compiled C++ > code) > -$ scripts/build > -$ sudo scripts/run.py -nv -e "/tools/ifconfig.so" > -``` > +- OSv can be debugged with gdb; for more details please read this > [wiki](https://github.com/cloudius-systems/osv/wiki/Debugging-OSv) > +- OSv kernel and application can be traced and profiled; for more > details > please read this > [wiki]( > https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py > ) > +- OSv comes with the admin/montioring REST API server; for more details > please read > [this]( > https://github.com/cloudius-systems/osv/wiki/Command-Line-Interface-(CLI)) > > and [that wiki > page](https://github.com/cloudius-systems/osv/wiki/Using-OSv-REST-API). > > -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/00000000000038fc980590705619%40google.com > . > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CANEVyjvTLaca2%2BJ2h6-c8o9mLD05CfK17_fSxxcYjtJW75MmUg%40mail.gmail.com.
