Re: [osv-dev] Performance of ffmpeg

[Roman Shaposhnik]

Old time ffmpeg hacker here -- please remember that ffmpeg has a
pretty convoluted build system that will disable accelerations (used
to do it even silently) if the environment is not setup quite right.
So... in order for this to be apples-to-apples it kind of needs to be
the same ELF.

Thanks,
Roman.

On Tue, Feb 16, 2021 at 2:06 PM Waldek Kozaczuk  wrote:
>
> Here is an output of something similar with 4 vCPUs:
>
> OSv v0.55.0-121-g3e898f4d
> eth0: 192.168.122.15
> Booted up in 170.53 ms
> Cmdline: /ffmpeg.so -i http://clips.vorwaerts-gmbh.de/VfE_html5.mp4 -c:v 
> libx265 -crf 28 -an -f mpegts tcp://192.168.122.1:12345
> ffmpeg version 4.2 Copyright (c) 2000-2019 the FFmpeg developers
>   built with gcc 8 (Ubuntu 8.3.0-6ubuntu1)
>   configuration: --disable-all --enable-ffmpeg --enable-small 
> --enable-avcodec --enable-avformat --enable-avfilter --enable-swresample 
> --enable-swscale --enable-decoder='h264,h265' 
> --enable-encoder='rawvideo,libx264,libx265,png' --enable-parser='h264,h265' 
> --enable-protocol='file,http' --enable-demuxer=mov 
> --enable-muxer='rawvideo,mp4,mpegts,image2' --enable-filter=scale 
> --enable-libx264 --enable-libx265 --enable-gpl
>   libavutil  56. 31.100 / 56. 31.100
>   libavcodec 58. 54.100 / 58. 54.100
>   libavformat58. 29.100 / 58. 29.100
>   libavfilter 7. 57.100 /  7. 57.100
>   libswscale  5.  5.100 /  5.  5.100
>   libswresample   3.  5.100 /  3.  5.100
> Guessed Channel Layout for Input Stream #0.1 : mono
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 
> 'http://clips.vorwaerts-gmbh.de/VfE_html5.mp4':
>   Metadata:
> major_brand : mp42
> minor_version   : 1
> compatible_brands: mp42avc1
> creation_time   : 2011-03-16T10:41:51.00Z
>   Duration: 00:01:00.08, start: 0.00, bitrate: 699 kb/s
> Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv, 
> smpte170m/smpte170m/bt709), 640x360, 633 kb/s, 24.04 fps, 24.04 tbr, 2500 
> tbn, 5k tbc (default)
> Metadata:
>   creation_time   : 2011-03-16T10:41:52.00Z
>   handler_name: Apple Video Media Handler
> Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, 62 kb/s 
> (default)
> Metadata:
>   creation_time   : 2011-03-16T10:41:53.00Z
>   handler_name: Apple Sound Media Handler
> Stream mapping:
>   Stream #0:0 -> #0:0 (h264 (native) -> hevc (libx265))
> Press [q] to stop, [?] for help
> x265 [info]: HEVC encoder version 2.9
> x265 [info]: build info [Linux][GCC 8.2.0][64 bit] 8bit+10bit+12bit
> x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX 
> FMA3 BMI2 AVX2
> x265 [info]: Main profile, Level-2.1 (Main tier)
> x265 [info]: Thread pool created using 4 threads
> x265 [info]: Slices  : 1
> x265 [info]: frame threads / pool features   : 2 / wpp(6 rows)
> x265 [warning]: Source height < 720p; disabling lookahead-slices
> x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
> x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
> x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 2
> x265 [info]: Keyframe min / max / scenecut / bias: 24 / 250 / 40 / 5.00
> x265 [info]: Lookahead / bframes / badapt: 20 / 4 / 2
> x265 [info]: b-pyramid / weightp / weightb   : 1 / 1 / 0
> x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
> x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
> x265 [info]: Rate Control / qCompress: CRF-28.0 / 0.60
> x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp 
> strong-intra-smoothing
> x265 [info]: tools: deblock sao
> Output #0, mpegts, to 'tcp://192.168.122.1:12345':
>   Metadata:
> major_brand : mp42
> minor_version   : 1
> compatible_brands: mp42avc1
> encoder : Lavf58.29.100
> Stream #0:0(eng): Video: hevc (libx265), yuv420p, 640x360, q=2-31, 24.04 
> fps, 90k tbn, 24.04 tbc (default)
> Metadata:
>   creation_time   : 2011-03-16T10:41:52.00Z
>   handler_name: Apple Video Media Handler
>   encoder : Lavc58.54.100 libx265
> Side data:
>   cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
> Input #0, mpegts, from 'tcp://0.0.0.0:12345?listen':
>   Duration: N/A, start: 1.483200, bitrate: N/A
>   Program 1
> Metadata:
>   service_name: Service01
>   service_provider: FFmpeg
> Stream #0:0[0x100]: Video: hevc (Main) (HEVC / 0x43564548), yuv420p(tv), 
> 640x360, 24 fps, 24.04 tbr, 90k tbn, 24.04 tbc
> Output #0, mp4, to '/tmp/test.mp4':
>   Metadata:
> encoder : Lavf58.45.100
> Stream #0:0: Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv), 
> 640x360, q=2-31, 24 fps, 24.04 tbr, 90k tbn, 90k tbc
> Stream mapping:
>   Stream #0:0 -> #0:0 (copy)
> Press [q] to stop, [?] for help
> frame= 1445 fps=102 q=-0.0 Lsize=2120kB time=00:00:59.98 bitrate= 
> 289.5kbits/s speed=4.22x
> video:1797kB audio:0kB subtitle:0kB other 

Re: [osv-dev] Re: Pip packages/using Nix

[Roman Shaposhnik]

So is this getting published some place? ;-)

Thanks,
Roman.

On Mon, Dec 14, 2020 at 9:00 AM Matthew Kenigsberg <
matthewkenigsb...@gmail.com> wrote:

> Thanks for putting so much effort into figuring that out! Really
> appreciate it, and glad to get it working!
>
> On Wednesday, December 9, 2020 at 3:45:30 PM UTC-7 Matthew Kenigsberg
> wrote:
>
>> That worked!!! Had to set -z relro -z lazy
>>
>> On Wednesday, December 9, 2020 at 12:30:55 PM UTC-7 jwkoz...@gmail.com
>> wrote:
>>
>>> Hi,
>>>
>>> Thanks for uploading the files. It definitely has helped me figure out
>>> the issue.
>>>
>>> In essence, all the .so files like libzfs.so are built with Full RELRO
>>> (run 'readelf -a libzfso.so | grep BIND_NOW) on Nix OS it looks like.
>>> Relatedly, some Linux distributions are setup to make gcc effectively use
>>> '-z now -z relro' when linking the libraries. On many others like Ubuntu or
>>> Fedora they are built with Partial RELRO - '-z relro' by default.
>>>
>>> As the libraries are loaded by OSv dynamic linker, all jump slot
>>> relocations are resolved eagerly (even if they are not used by the code
>>> later) if those libraries are marked as 'Full RELRO' (bind_now = true). For
>>> non-'Full RELRO' cases, the jump slot relocations are resolved lazily
>>> whenever they are accessed 1st time and are handled by 'void*
>>> object::resolve_pltgot(unsigned index)` which writes resolved function
>>> symbol address in GOT.
>>>
>>> The problem with Full-RELRO is that if we cannot find a symbol because
>>> for example it is not implemented by OSv or is not visible at *this
>>> point of linking* we simply ignore it hoping that it will never be used
>>> or resolved later. If it is used later, the resolve_pltgot() is called, and
>>> if the symbol is found (because the library containing the symbol has been
>>> loaded since) we crash because we trying to write to the part of memory -
>>> GOT - that has been since read-only protected.
>>>
>>> Why does this happen exactly?
>>>
>>> So here is the symbol *bsd_getmntany *not found at the address you were
>>> getting original fault at (after adding extra debug statements):
>>> ELF [tid:28, mod:3, /libzfs.so]: arch_relocate_jump_slot,
>>> addr:0x10040ca0
>>> /libzfs.so: ignoring missing symbol *bsd_getmntany //Would have been 
>>> *0x10040ca8
>>> which match what page fault reports
>>> ELF [tid:28, mod:3, /libzfs.so]: arch_relocate_jump_slot,
>>> addr:0x10040cb0
>>>
>>> Please note that both mkfs.so and zfs.so depend on libzfs.so. Per
>>> command line, OSv loads and executes the apps sequentially. So the mkfs.so
>>> is first and the dynamic linker will load libzfs.so and relocate and
>>> eagerly resolve all symbols and fix the permissions on libzfs.so. One of
>>> the symbols in libzfs.so is *bsd_getmntany* which is actually part of
>>> zfs.so which is left unresolved (see missing warning above).
>>>
>>> After mkfs.so, OSv gets to zfs.so and it processes it and executes and
>>> some of the code in zfs.so tries to invoke *bsd_getmntany *which gets
>>> dynamically resolved and found by resolve_pltgot() BUT when it tries to
>>> write to GOT it gets page fault.
>>>
>>> Having said all that I am sure what exactly the problem is:
>>> A) Invalid or abnormal dependency between libzfs.so, mkfs.so and zfs.so
>>> which effectively prevents those to function properly if build with Full
>>> RELRO (would that work on Linux)?
>>> B) True limitation of OSv linker which should handle correctly such
>>> scenario.
>>>
>>> For now, the easiest solution (might be true if A is true) is to simply
>>> force building those libraries with Partial RELRO like in this patch:
>>>
>>> diff --git a/Makefile b/Makefile
>>> index d1597263..d200dde8 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -345,7 +345,7 @@ $(out)/%.o: %.s
>>> $(makedir)
>>> $(call quiet, $(CXX) $(CXXFLAGS) $(ASFLAGS) -c -o $@ $<, AS $*.s)
>>>
>>> -%.so: EXTRA_FLAGS = -fPIC -shared
>>> +%.so: EXTRA_FLAGS = -fPIC -shared -z relro
>>>  %.so: %.o
>>> $(makedir)
>>> $(q-build-so)
>>>
>>> Please let me know if it works,
>>> Waldek
>>>
>>> PS. Also verify that running ' readelf -a libzfso.so | grep BIND_NOW'
>>> does not show anything anymore.
>>>
>>> On Tuesday, December 8, 2020 at 5:08:18 PM UTC-5 Matthew Kenigsberg
>>> wrote:
>>>
 Not completely sure where libgcc_s.so.1 is coming from, but I uploaded what
 I have in
 /nix/store/vran8acwir59772hj4vscr7zribvp7l5-gcc-9.3.0-lib/lib/libgcc_s.so.1:
 https://drive.google.com/drive/folders/1rM6g-FrzwFpuHr2wX9-J21DzSjyQXGg2

 Get a different error if I comment out core/elf.cc:1429:

 (gdb) bt
 #0 0x4039eef2 in processor::cli_hlt () at
 arch/x64/processor.hh:247
 #1 arch::halt_no_interrupts () at arch/x64/arch.hh:48
 #2 osv::halt () at arch/x64/power.cc:26
 #3 0x4023c73f in abort (fmt=fmt@entry=0x40645aff "Aborted\n")
 at runtime.cc:132
 #4 0x40202989 in abort () at 

Re: [osv-dev] Re: Pip packages/using Nix

[Roman Shaposhnik]

On Tue, Nov 24, 2020 at 8:03 AM Waldek Kozaczuk  wrote:
>
> Hey,
>
> Send a patch with a new app that could demonstrate it, please, if you can. I 
> would like to see it. Sounds like a nice improvement.

FWIW: I'd love to see it too -- been meaning to play with Nix and this
gives me a perfect excuse ;-)

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CA%2BULb%2BseTV8sOL7Sq4gfUga%3Dy7wuGirWhoHoBStsaCDsji5-cA%40mail.gmail.com.

Re: [osv-dev] Next releases - 0.56 and onward - proposal

[Roman Shaposhnik]

This is such an impressive list! Would love to see it all!

Thanks,
Roman.

On Sat, Jun 6, 2020 at 9:45 AM Waldek Kozaczuk  wrote:
>
> Hi,
>
> Here is a list of things, not necessarily in any importance order, I am 
> proposing we try to accomplish for next releases:
>
> Prune the list of issues
>
> Close anything that is irrelevant or possibly completed
>
> One subcategory I can think of, are issues about improving java wrapper like 
> this one - https://github.com/cloudius-systems/osv/issues/169. Given we can 
> run unmodified /usr/bin/java without need of a wrapper and given the wrapper 
> (java.so) does not support (or is even compatible) with many features of Java 
> 9 and higher, I suggest we close any issues like 169 which are about 
> improving java.so - I do not think it is worth any time investment (the only 
> real benefit of java.so is ballooning feature which is broken - 
> https://github.com/cloudius-systems/osv/issues/796 and 
> https://github.com/cloudius-systems/osv/issues/1038 [duplicate of 796?] - but 
> if anyone is interested in fixing it I think we might support it without 
> java.so anyway).
>
> Better annotate existing issues and especially select ones as '"good first 
> issue" (see 
> https://github.blog/2020-01-22-browse-good-first-issues-to-start-contributing-to-open-source/)
>
> Better support Linux executables built for Musl-based distributions like 
> Alpine (see https://wiki.musl-libc.org/projects-using-musl.html)
>
> These may work even better given OSv libc implementation is to great extent 
> based on musl
>
> Tooling
>
> Improve capstan
>
> Better support QEMU on OSX (new accelerators)
> Support pulling artifacts from GitHub nightly releases repo
> Support running Firecracker, QEMU microvm
> Support Virtio-FS
>
> Create plugins to deploy Java, Node, etc apps
>
> IntelliJ
> Gradle
> Maven
>
> Run Docker images (possibly musl-based like Alpine are better)
>
> There is already existing example - 
> https://github.com/cloudius-systems/osv-apps/tree/master/openjdk12-jre-from-docker
>
> Review the lazy app thread stack patch aimed to reduce memory consumption
>
> https://groups.google.com/d/msg/osv-dev/tZnwiScmjZY/GkY0hV9EAwAJ
>
> Modularization/"Librarization"
>
> See the discussion - 
> https://groups.google.com/d/msg/osv-dev/BHwN6Stm3n4/arvdOwl3AAAJ
> Hide most symbols in the kernel
>
> Issue #97 - Be more selective on symbols exported from the kernel
> See related discussion - 
> https://groups.google.com/d/msg/osv-dev/fs-jmyaXY_E/vWUk-wDDAAAJ
>
> Reduce kernel size:
>
> See the discussion - 
> https://groups.google.com/d/msg/osv-dev/hCYGRzytaJ4/D23S_ibNAgAJ and 
> https://groups.google.com/d/msg/osv-dev/JsJfnhTX_EY/PKUvYm9-AQAJ
>
> Expose C++ APIs as C to hide C++ completely
>
> Keep improving Virtio-FS (#1079)
>
> Integrate with pagecache
> Improve DAX
>
> Improve ARM support
>
> See this Firecracker patch - 
> https://groups.google.com/d/msg/osv-dev/3TrFxpW9Bhk/AdilzZT5AQAJ
> Improve run.py
>
> Reach out to Spirent to see if any more networking related bugs fixed can be 
> brought over from the https://github.com/SpirentOrion/osv fork
> Add support of Virtio GPU/VGA
> Upgrade musl
>
> I kind of remember Geraldo Netto started on this
>
> Test newer runtimes
>
> Golang >= 1.13
> Java > 12
> DENO 1.0
>
> Support .NET Core executables
>
> See what needs to be finished in this thread - 
> https://groups.google.com/d/msg/osv-dev/k69cHw7qvTg/9NqPq70IAwAJ
>
> Refresh look and feel of http://osv.io
>
> Other interesting existing issues:
>
> Better deal with small amount of memory - 
> https://github.com/cloudius-systems/osv/issues/1013
> #28 - Optimize atomic operations on UP (single-VCPU)
> #1016 - Add support for smb2/3 (samba) file system T
>
> There is already a patch sent to the list - 
> https://groups.google.com/d/msg/osv-dev/QToC22-QHiA/ejSOMGe7BwAJ
>
> #1009 - Make ZFS optional as a shared library
> #212 - Support static executables
> #1069 - Add vsock support
> #1041 - Idea for easily creating OSv images from list of Ubuntu packages
>
> does manifest_from_host.sh provide most of it?
>
> Fix networking bugs
>
> https://github.com/cloudius-systems/osv/issues/420
> https://github.com/cloudius-systems/osv/issues/454
>
> Faster VFS - https://github.com/cloudius-systems/osv/issues/450
> https://github.com/cloudius-systems/osv/issues/664
> https://github.com/cloudius-systems/osv/issues/589
> Improve XEN support
>
> https://github.com/cloudius-systems/osv/issues/498
> https://github.com/cloudius-systems/osv/issues/345
>
> https://github.com/cloudius-systems/osv/issues/264
> https://github.com/cloudius-systems/osv/issues/235
> Speed-up realloc() by mremap() - 
> https://github.com/cloudius-systems/osv/issues/184
> “Steal-time” to improve scheduler - 
> https://github.com/cloudius-systems/osv/issues/181
> Investigate virtio-net “slow path” - 
> https://github.com/cloudius-systems/osv/issues/177
> Try GOLD linker - https://github.com/cloudius-systems/osv/issues/151
>
> Gold 

Re: [osv-dev] OSv boots and runs simple hello world on Firecracker ARM edition ... on Raspberry PI 4

[Roman Shaposhnik]

This is super awesome, Waldek! I'm now inspired to run OSv as a VM
backed by Firecracker on Project EVE/RPi! ;-)

Btw, can you share your osv-config.json and also details of what
version of Firecracker you used?

Better yet -- if there's a github repo with all the bits and pieces
required to replicate your setup -- that'd be worthy of a gift of a
beer (post COVID tho!) ;-)

Thanks,
Roman.

On Thu, May 21, 2020 at 1:58 PM Waldek Kozaczuk  wrote:
>
>
> ubuntu@ubuntu-rasp4:~$ uname -a
>
> Linux ubuntu-rasp4 5.3.0-1025-raspi2 #27-Ubuntu SMP Fri May 8 08:32:04 UTC 
> 2020 aarch64 aarch64 aarch64 GNU/Linux
>
> ubuntu@ubuntu-rasp4:~$ ./firecracker-v0.21.1-aarch64 --no-api --config-file 
> ./osv-config.json
>
> OSv v0.55.0-9-g840428ad
>
> PSCI: version 65536.0 detected.
>
> setup_arm_clock() ENTERED, lr=800d54e0
>
> arm_clock(): frequency read as 0337f980
>
> interrupt_table::interrupt_table() ENTERED, lr=800a8d1c
>
> gic_driver::init_cpu() ENTERED, lr=80201c80
>
> CPU interface enabled.
>
> gic_driver::init_dist() ENTERED, lr=80201c8c
>
> number of supported IRQs: 0080
>
> interrupt table: gic driver created.
>
> registered IRQ id=0004
>
> registered IRQ id=001b
>
> registered IRQ id=
>
> Premain complete!
>
> smp_launch ENTERED, lr=800d709c
>
> Booted up in 0.00 ms
>
> Cmdline: console=ttyS0 --verbose --nomount --maxnic=0 /tools/hello.so 
> earlycon=uart,mmio,0x40001000
>
> faulting address 1001fea0
>
> faulting address 1328
>
> faulting address 1002
>
> faulting address 10050018
>
> faulting address 100301c8
>
> faulting address 1004fe00
>
> Hello from C code
>
> 2020-05-21T20:24:04.080765641 
> [anonymous-instance:ERROR:src/vmm/src/vstate.rs:951] Unexpected exit reason 
> on vcpu run: SystemEvent
>
>
> In general it took me a bit of research as I am not really familiar with ARM 
> architecture and even reading the assembly was a bit a challenge to say the 
> least. And then debugging without debugger and any console (:-( .. so not 
> debug() for long time. But all in all it was not too bad and the changes that 
> I had to make to OSv are in my opinion much smaller and easier comparing to 
> x86_64.
>
> Below you will see the "hack-patch" showing what changes I had to make. 
> Logically, following things had to be changed:
>
> The most important thing was to move kernel from 0x4000 (1GB) to 
> 0x8000 (2GB) which required changing one line in Makefile (see below) and 
> changing boot paging table to map the 2GB-3GB area of memory; only then I 
> could actually start debugging :-) I wonder if it will also work on QEMU/KVM 
> - possinly qemu boot loader will inspect ELF and place it accordingly in 
> memory; firecracker does not read ELF and simply places it at 2GB
> To get console working I guessed that that need to create equivalent class 
> for isa_serial_console but communicating over mmio (see 
> mmio_isa_serial_console.hh/cc) which in essence invokes mmio_set*/mmio_get* 
> (it would be nice to extract common code somehow - suggestions welcome)
> Some more trivial changes - for now mostly disabling things - for now quite 
> disorganized
>
>
> diff --git a/Makefile b/Makefile
> index db3c68cf..ffd570a5 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -354,6 +354,7 @@ tools := tools/mkfs/mkfs.so tools/cpiod/cpiod.so
>  $(out)/tools/%.o: COMMON += -fPIC
>
>  tools += tools/uush/uush.so
> +tools += tools/uush/hello.so
>  tools += tools/uush/ls.so
>  tools += tools/uush/mkdir.so
>
> @@ -451,8 +452,8 @@ endif # x64
>
>  ifeq ($(arch),aarch64)
>
> -kernel_base := 0x4008
> -kernel_vm_base := 0x4008
> +kernel_base := 0x8008
> +kernel_vm_base := 0x8008
>  app_local_exec_tls_size := 0x0
>
>  include $(libfdt_base)/Makefile.libfdt
> @@ -816,6 +817,7 @@ drivers += drivers/xenplatform-pci.o
>  endif # x64
>
>  ifeq ($(arch),aarch64)
> +drivers += drivers/mmio-isa-serial.o
>  drivers += drivers/pl011.o
>  drivers += drivers/xenconsole.o
>  drivers += drivers/virtio.o
> diff --git a/arch/aarch64/arch-dtb.cc b/arch/aarch64/arch-dtb.cc
> index b59f1dcc..cd0719e8 100644
> --- a/arch/aarch64/arch-dtb.cc
> +++ b/arch/aarch64/arch-dtb.cc
> @@ -225,7 +225,8 @@ bool dtb_get_gic_v2(u64 *dist, size_t *dist_len, u64 
> *cpu, size_t *cpu_len)
>  if (!dtb)
>  return false;
>
> -node = fdt_node_offset_by_compatible(dtb, -1, "arm,cortex-a15-gic");
> +//node = fdt_node_offset_by_compatible(dtb, -1, "arm,cortex-a15-gic");
> +node = fdt_node_offset_by_compatible(dtb, -1, "arm,gic-400");
>  if (node < 0)
>  return false;
>
> @@ -613,11 +614,13 @@ void  __attribute__((constructor(init_prio::dtb))) 
> dtb_setup()
>  abort("dtb_setup: failed to parse cpu mpid.\n");
>  }
>
> +//TODO: Parse PCI only if required and
> +// abort if missing only if required
>  dtb_timer_irq = dtb_parse_timer_irq();
>  

Re: [osv-dev] OSv dockerfile

[Roman Shaposhnik]

This is pretty sweet! Can this be added to the official OSv repo as well?

Thanks,
Roman.

On Fri, Aug 2, 2019 at 9:04 AM Waldek Kozaczuk  wrote:

> Hi,
>
> As part of the effort to upgrade OpenSSL, in order to test the changes, I
> have created a couple of Docker files allowing to create full OSv build
> setup. They are part of my repo -
> https://github.com/wkozaczuk/docker_osv_builder. In essence each of the
> two files (Dockerfile.Fedora and Dockerfile.Ubuntu) builds an image from
> Fedora or Ubuntu base, adds python and git, clones OSv repo and eventually
> calls ./scripts/setup.py to install all packages.
>
> I think they could be pretty useful for others so I was wondering if
> should simply reference my repo somewhere on OSV wiki or check in the file
> to OSv repo somewhere under scripts. Any ideas?
>
> Also, I have not much experience with publishing docker images to
> dockerhub. Can it be done for free? Would it be useful to build those and
> publish there? I remember there was something called "router" in the past.
> But that probably is obsolete now.
>
> Waldek
>
> Here is how one of the Docker files looks:
>
> #FROM fedora:27
> FROM fedora:29
>
> ENV DEBIAN_FRONTEND noninteractive
> ENV TERM=linux
>
> COPY ./etc/keyboard /etc/default/keyboard
> COPY ./etc/console-setup /etc/default/console-setup
>
> RUN yum install -y git python2 file which
>
> #
> # PREPARE ENVIRONMENT
> #
>
> # - prepare directories
> RUN mkdir /git-repos /result
> # - clone OSv
> WORKDIR /git-repos
> RUN git clone https://github.com/cloudius-systems/osv.git
> WORKDIR /git-repos/osv
> RUN git submodule update --init --recursive
> RUN scripts/setup.py
>
> RUN update-alternatives --set java /usr/lib/jvm/java-1.8.0-openjdk/bin/java
>
> CMD /bin/bash
>
> #
> # NOTES
> #
> # Build this container with:
> # docker build -t osv/builder-fedora -f Dockerfile.Fedora .
> #
> # Run this container with:
> # docker restart ID (from docker ps -a) && docker attach ID
> #
> # To open in another console
> # docker exec -it ID /bin/bash
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/32591560-fff6-4b22-a27e-cb43189225d3%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CA%2BULb%2BtCL%3DQqf8bg4%2BCd88RezKyx1kvU_E0KO0xszm0HHH8Aqg%40mail.gmail.com.

Re: [osv-dev] OSv Linux compatibility

[Roman Shaposhnik]

Well, there's that whole "you need to convert your a.out into a.so"
thing -- so I guess it is kind of a porting

On Wed, Apr 10, 2019 at 11:23 AM Waldek Kozaczuk  wrote:
>
> I came across another unikernels-overview-type-of-writeup project on github - 
> https://github.com/cetic/unikernels/blob/master/README.md.
>
> It has this description:
> “OSv is particular; it allows porting various applications written in various 
> languages to a unikernel by integrating some kernel functions built-in. 
> Therefore, it is not as light as a fully compiled unikernel, however it 
> provides support for many languages including Java, C, C++, Node and Ruby. ”
>
> How accurate is it? What does “porting” mean? Given OSv can run unmodified 
> JVM so?
>
> Do you think there is some confusion in this area? Maybe outdated docs?
>
> --
> You received this message because you are subscribed to the Google Groups 
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to osv-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [osv-dev] Does OSv have its own scheduler?

[Roman Shaposhnik]

FWIW: KVM is a type-2 hypervisor, Xen (and ACRN and a few others) are
type-1. Both have advantages and disadvantages.

For example, a type-2 hypervisor is very difficult to tune for running RT
OSes while ACRN (and even Xen lately -- especially on ARM) do it with no
sweat.

Thanks,
Roman.


On Mon, Feb 18, 2019 at 11:21 PM Joe Duarte  wrote:

>
>
> On Monday, February 18, 2019 at 1:43:29 PM UTC-8, Nadav Har'El wrote:
>>
>>
>>
>> KVM and Xen are *hypervisors*, i.e., software which allows running
>> multiple virtual-machines on one physical machine.
>> The "Linux scheduler" and "Xen scheduler" you mention are implementation
>> details of that hypervisor, and has nothing to do with OSv.
>>
>>
> Right, I know all that. What I was saying is that KVM is supposed to have
> an *advantage* over Xen because there's only one scheduler, because while
> KVM is technically a hypervisor, it's in the Linux kernel (not underneath
> it like Xen). I got this point from Brendan Gregg, a performance engineer
> at Joyent and now Netflix. One place he touches on it is in this doc
> :
> "With KVM, the hypervisor is a kernel module (kvm) which is scheduled by
> the OS scheduler. It can be tuned using the usual OS kernel scheduler
> classes, policies and priorities. The I/O path takes fewer steps than Xen."
>
> I took this as having implications for OSv since OSv also runs on top of
> Xen with, apparently, its own scheduler. (I don't know if it's restricted
> to Xen anymore.) I don't know if or how the schedulers interact. Is OSv I/O
> (on Xen) faster than KVM VM I/O? Table 1 from the paper might bear on it,
> but it's five years old and I'm not clear as to what the Linux setup was
> (was it Fedora 20 running KVM with a Fedora 20 guest on top?).
>
> JD
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Next (0.53.0 and onwards) OSv releases roadmap proposal

[Roman Shaposhnik]

On Tue, Nov 13, 2018 at 11:26 AM Geraldo Netto  wrote:
>
> Hello Roman/Friends,
>
> Roman, I'm having the freedom to share this message with the list once I'm 
> not an expert in gvisor
> Actually, the idea is not to integrate gvisor, but instead take the list of 
> implemented syscalls and check if we implemented them in OSv or not

That actually makes tons of sense. Btw, I had a chat with gvisor folks
last week and tried
to convince them to open source their test suite that they are using
internally to track
the level of compatibility with various Linux apps. Suffice it to say,
Google ppl. were... elusive

> But theoretically, gvisor is written in go and go can be interfaced with 
> assembly/c/c++/...
> So, theoretically, we might be able to replace OSv kernel by gvisor but that 
> would have other implications as far as I know
> eg: gvisor is a user space implementation of linux kernel which means we 
> would still need to provide some linux ABI somehow
> Maybe someone from the list might be able to correct me if I'm saying some 
> no-sense...

That's what I assumed when I asked the question -- *theoretically*
this could be done since
Go can be linked with C++ just fine as a shared object and then you
can route syscalls
to that.

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Next (0.53.0 and onwards) OSv releases roadmap proposal

[Roman Shaposhnik]

I've got to say: this is a really awesome list!
On Tue, Nov 13, 2018 at 10:54 AM Waldek Kozaczuk  wrote:
>
> This is somewhat revised version of the roadmap I sent couple of months ago. 
> I think that the main premise of it which I wrote in the original version 
> still stands:
>
> "In my opinion, if OSv is to become more relevant than it is now, its primary 
> target should be a platform for running stateless and serverless apps. 
> Therefore it needs to become leaner (memory usage and kernel size), a little 
> more modular and boot faster while preserving its functionality in terms of 
> Linux/POSIX ABI (and possibly expanding where it makes sense). That is not 
> say that we should not improve support for stateful apps like mysql or 
> elasticsearch which would require optimizations to VFS and possibly ZFS which 
> I list below."
>
> OBJECTIVES
>
> Make OSv more modular
>
> ZFS and commands/runscript functionality should be moved as optional modules 
> in form of shared libraries
> The key building block to achieve it should be an improved ROFS intended as a 
> default filesystem to load code from
>
> Optimize kernel size
>
> My experiments show that kernel (loader-stripped.elf) can be as small as 
> 5.1MB after ZFS and program options are compiled out (and there is still room 
> for improvements)
>
> Optimize boot time
>
> OSv can execute under 50ms on hyperkit and under 100ms on QEMU with qboot
> Smaller kernel should reduce boot time
>
> Optimize memory usage
>
> My recent investigation about memory allocation is OSv shows that it should 
> be possible and quite easy to reduce memory utilization in many areas
> The aggressive but quite realistic goal should be an ability to run OSv with 
> simple C app using no more than 10MB of RAM
> Smaller kernel should also reduce memory usage
>
>
> MORE IMPORTANT
> File systems
>
> ROFS
>
> [FASTER BOOT] Add compression - 
> https://github.com/cloudius-systems/osv/issues/978
> [LESS MEMORY] Avoid creating another copy when mmap-ing ROFS file - 
> https://github.com/cloudius-systems/osv/issues/979
>
> ZFS
>
> [MODULARITY, SMALLER KERNEL, FASTER BOOT] move all ZFS code in kernel into 
> libzfs.so
>
> Make ZFS an optional library - 
> https://github.com/cloudius-systems/osv/issues/1009
>
> RAMFS
>
> [SPEED, LESS MEMORY] Fix slow write/append of files on RAMFS - 
> https://github.com/cloudius-systems/osv/issues/884
> Eliminate BootFS
>
> 9pFS
>
> [ENHANCEMENTS] https://github.com/cloudius-systems/osv/issues/1008
>
>
> Optimize memory usage
>
> Wasted memory in early (pre-SMP-enabled) malloc (possibly save 6MB of 
> memory!!!) - https://github.com/cloudius-systems/osv/issues/270
> Make allocations < 16 bytes more space efficient - 
> https://github.com/cloudius-systems/osv/issues/1011
> Improve physical memory utilization by using memory below 2MB - 
> https://github.com/cloudius-systems/osv/issues/1012
> Make L1/L2 memory pool sizes self-configurable depending on physical memory 
> available - https://github.com/cloudius-systems/osv/issues/1013
> Consider more space-efficient allocation between 1024 and 4096 bytes - 
> https://github.com/cloudius-systems/osv/issues/1000
>
>
> Improve modularity
>
> Remove boost programs_options from kernel by rewriting 
> loader.cc::parse_options() to use getopt_long() and extract 
> commands.cc:parse_command_line(const std::string line, bool& ok) as a 
> optional library commands.so; see 
> https://github.com/cloudius-systems/osv/issues/1014 for details
>
>
> Optimize kernel size
>
> Be more selective on symbols exported from the kernel - 
> https://github.com/cloudius-systems/osv/issues/97
>
> Remove/hide symbols
>
> Investigate what else can be discarded using bloaty
>
> eliminate some debug strings in BSD code tree that should further reduce 
> .rodata section
> remove/hide symbols (reduces ELF .dynsym (others?) section) - 
> https://github.com/cloudius-systems/osv/issues/97
> reduce size of other sections in loader.elf
>
>
> Other
>
> Make more programs run without having to re-compile (on ubuntu already pies)
>
> look at adding missing symbols to make coreutils (ls, find) programs run 
> (sent email about it)
>
> Stop using libraries and headers from external - 
> https://github.com/cloudius-systems/osv/issues/743 and related … 
> https://github.com/cloudius-systems/osv/issues/687
> Assertion failed: timestamp >= _last (timer-set.hh: expire) - 
> https://github.com/cloudius-systems/osv/issues/382
>
>
> COMPLETE BUT NOT COMMITTED PATCHES
>
> [SECURITY] Read-only REST API
>
> https://github.com/cloudius-systems/osv/issues/820
>
> mprotect() syscall stack
>
> we could commit it as is but then it would syscall stack use more memory (1 
> page per thread) until we fix this issue - 
> https://github.com/cloudius-systems/osv/issues/1000
>
> full revival of multiboot
>
> there may be still controversial issues resolved before this patch is ready 
> to be merged
>
>
> LESS IMPORTANT
>
> [ENHANCEMENTS] Support new hypervisors
>
> 

Re: About OSv support for docker image using 9pfs

[Roman Shaposhnik]

On Wed, Oct 31, 2018 at 11:36 AM Waldek Kozaczuk  wrote:
>
> If anyone is interested I have created an issue that describes what it would 
> take to
> implement v9fs - https://github.com/cloudius-systems/osv/issues/1008. It does 
> not
> seem to be difficult nor a big effort but based on what I have googled it is 
> not clear
> if it performs better than NFS that we already implement.

I think the biggest advantage of 9p over NFS is that you can use this
kind of a transport:
   http://www.linux-kvm.org/page/9p_virtio
instead of the network.

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About OSv support for docker image using 9pfs

[Roman Shaposhnik]

On Thu, Sep 27, 2018 at 5:42 PM, Qixuan Wu  wrote:
>
>
> 在 2018年9月21日星期五 UTC+8下午1:11:40，Roman Shaposhnik写道：
>>
>> On Thu, Sep 20, 2018 at 10:00 PM, Qixuan Wu  wrote:
>> >
>> >
>> > 在 2018年9月21日星期五 UTC+8上午1:59:50，Roman Shaposhnik写道：
>> >>
>> >> On Thu, Sep 20, 2018 at 5:10 AM, Qixuan Wu  wrote:
>> >> >
>> >> >
>> >> > 在 2018年9月19日星期三 UTC+8下午12:44:54，Roman Shaposhnik写道：
>> >> >>
>> >> >> On Tue, Sep 18, 2018 at 7:27 PM, Qixuan Wu 
>> >> >> wrote:
>> >> >> >
>> >> >> >
>> >> >> > 在 2018年9月19日星期三 UTC+8上午10:00:36，Roman Shaposhnik写道：
>> >> >> >>
>> >> >> >> On Tue, Sep 18, 2018 at 5:48 PM, Qixuan Wu 
>> >> >> >> wrote:
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > 在 2018年9月19日星期三 UTC+8上午5:56:58，Nadav Har'El写道：
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On Tue, Sep 18, 2018 at 11:59 AM, Qixuan Wu
>> >> >> >> >> 
>> >> >> >> >> wrote:
>> >> >> >> >>>
>> >> >> >> >>> Hi all,
>> >> >> >> >>>
>> >> >> >> >>>We have some requirement to support standard docker OCI
>> >> >> >> >>> image.
>> >> >> >> >>>
>> >> >> >> >>>Currently OSv did not support 9pfs. Anyone are thinking of
>> >> >> >> >>> this
>> >> >> >> >>> or
>> >> >> >> >>> not
>> >> >> >> >>> ?
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> Supporting 9p and virtfs has been on our wishlist for quite
>> >> >> >> >> some
>> >> >> >> >> time
>> >> >> >> >> (see
>> >> >> >> >> https://github.com/cloudius-systems/osv/issues/210) but nobody
>> >> >> >> >> is
>> >> >> >> >> actively
>> >> >> >> >> working on it.
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > This is a very important feature. If we can do it, it will be
>> >> >> >> > easy
>> >> >> >> > can
>> >> >> >> > support the compatibility with the OCI docker image. Some
>> >> >> >> > docker
>> >> >> >> > container
>> >> >> >> > can securely run inside OSv. That we can call it is another
>> >> >> >> > secure
>> >> >> >> > container
>> >> >> >> > solution, like kata container, gVisor.
>> >> >> >>
>> >> >> >> FWIW: I was looking to prototype 9pfs support for a hackathon
>> >> >> >> back
>> >> >> >> in
>> >> >> >> 2014.
>> >> >> >> It looked pretty manageable, frankly. Not sure I can help with
>> >> >> >> coding
>> >> >> >> it up again,
>> >> >> >> but I can definitely help with whatever else may be needed.
>> >> >> >
>> >> >> >
>> >> >> > I am searching the virtfs/9p source code from linux or some bsd
>> >> >> > code.
>> >> >> > But seems freebsd/netbsd still not support it. I only want the
>> >> >> > client,
>> >> >> > because qemu
>> >> >> > has the server already.
>> >> >> >
>> >> >> > Do you have any some suggestion for the code from where to port ?
>> >> >>
>> >> >> If what I think you're trying to do is, indeed, what you are
>> >> >> actually
>> >> >> trying
>> >> >> to do, I'd look here:  https://lwn.net/Articles/716582/
>> >> >>
>> >> >> Thanks,
>> >> >> Roman.
>> >> >>
>> >> >> P.S. Stefano is a good friend of mine -- so if that content seems
>> >> >> useful
>> >> >> to you I can do a direct intro.
>> >> >
>> >> >
>> >> > I think I will make the NFS (and + vsock later) as the rootfs now.
>> >> > Because
>> >> > 9p protocal and the code is too old, and maybe nobody want to
>> >> > maintain
>> >> > it.
>> >> > NFS is very popular now.
>> >>
>> >> I tend to disagree. This is how we ended up implementing Docker support
>> >> for Xen (which I think is very close to your usecase for OSv -- at
>> >> least conceptually):
>> >>
>> >>
>> >> https://github.com/rkt/stage1-xen/blob/master/build/fedora/RUNNING_STAGE1_XEN.md
>> >
>> >
>> > I saw it's already implement the 9pfs in the link
>> > Why you tend to disagree to implement docker support of OSv?
>>
>> I don't. I disagreed with your assumption that "Because 9p protocal and
>> the
>> code is too old, and maybe nobody want to maintain it."
>
>
> Sorry, maybe the description is not correct. What I saw is many people is
> developing
> other solution like nfs/vsock or FUSE_over_virtio on kata container, they
> are
> abandoning 9pfs.
>
> So maybe it's not worth porting 9pfs with so much efforts.

In my experience NFS is definitely not a good solution for something like
this unless you're willing to commit to a full stateful NFSv4 implementation.
Haven't seen any performance data on FUSE_over_virtio -- but until I do,
I'd personally stick with a much more predictable peformance implications
of 9pfs.

Just my 2c.

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About OSv support for docker image using 9pfs

[Roman Shaposhnik]

On Thu, Sep 20, 2018 at 10:00 PM, Qixuan Wu  wrote:
>
>
> 在 2018年9月21日星期五 UTC+8上午1:59:50，Roman Shaposhnik写道：
>>
>> On Thu, Sep 20, 2018 at 5:10 AM, Qixuan Wu  wrote:
>> >
>> >
>> > 在 2018年9月19日星期三 UTC+8下午12:44:54，Roman Shaposhnik写道：
>> >>
>> >> On Tue, Sep 18, 2018 at 7:27 PM, Qixuan Wu  wrote:
>> >> >
>> >> >
>> >> > 在 2018年9月19日星期三 UTC+8上午10:00:36，Roman Shaposhnik写道：
>> >> >>
>> >> >> On Tue, Sep 18, 2018 at 5:48 PM, Qixuan Wu 
>> >> >> wrote:
>> >> >> >
>> >> >> >
>> >> >> > 在 2018年9月19日星期三 UTC+8上午5:56:58，Nadav Har'El写道：
>> >> >> >>
>> >> >> >>
>> >> >> >> On Tue, Sep 18, 2018 at 11:59 AM, Qixuan Wu 
>> >> >> >> wrote:
>> >> >> >>>
>> >> >> >>> Hi all,
>> >> >> >>>
>> >> >> >>>We have some requirement to support standard docker OCI
>> >> >> >>> image.
>> >> >> >>>
>> >> >> >>>Currently OSv did not support 9pfs. Anyone are thinking of
>> >> >> >>> this
>> >> >> >>> or
>> >> >> >>> not
>> >> >> >>> ?
>> >> >> >>
>> >> >> >>
>> >> >> >> Supporting 9p and virtfs has been on our wishlist for quite some
>> >> >> >> time
>> >> >> >> (see
>> >> >> >> https://github.com/cloudius-systems/osv/issues/210) but nobody is
>> >> >> >> actively
>> >> >> >> working on it.
>> >> >> >
>> >> >> >
>> >> >> > This is a very important feature. If we can do it, it will be easy
>> >> >> > can
>> >> >> > support the compatibility with the OCI docker image. Some docker
>> >> >> > container
>> >> >> > can securely run inside OSv. That we can call it is another secure
>> >> >> > container
>> >> >> > solution, like kata container, gVisor.
>> >> >>
>> >> >> FWIW: I was looking to prototype 9pfs support for a hackathon back
>> >> >> in
>> >> >> 2014.
>> >> >> It looked pretty manageable, frankly. Not sure I can help with
>> >> >> coding
>> >> >> it up again,
>> >> >> but I can definitely help with whatever else may be needed.
>> >> >
>> >> >
>> >> > I am searching the virtfs/9p source code from linux or some bsd code.
>> >> > But seems freebsd/netbsd still not support it. I only want the
>> >> > client,
>> >> > because qemu
>> >> > has the server already.
>> >> >
>> >> > Do you have any some suggestion for the code from where to port ?
>> >>
>> >> If what I think you're trying to do is, indeed, what you are actually
>> >> trying
>> >> to do, I'd look here:  https://lwn.net/Articles/716582/
>> >>
>> >> Thanks,
>> >> Roman.
>> >>
>> >> P.S. Stefano is a good friend of mine -- so if that content seems
>> >> useful
>> >> to you I can do a direct intro.
>> >
>> >
>> > I think I will make the NFS (and + vsock later) as the rootfs now.
>> > Because
>> > 9p protocal and the code is too old, and maybe nobody want to maintain
>> > it.
>> > NFS is very popular now.
>>
>> I tend to disagree. This is how we ended up implementing Docker support
>> for Xen (which I think is very close to your usecase for OSv -- at
>> least conceptually):
>>
>> https://github.com/rkt/stage1-xen/blob/master/build/fedora/RUNNING_STAGE1_XEN.md
>
>
> I saw it's already implement the 9pfs in the link
> Why you tend to disagree to implement docker support of OSv?

I don't. I disagreed with your assumption that "Because 9p protocal and the
code is too old, and maybe nobody want to maintain it."

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About OSv support for docker image using 9pfs

[Roman Shaposhnik]

On Thu, Sep 20, 2018 at 5:10 AM, Qixuan Wu  wrote:
>
>
> 在 2018年9月19日星期三 UTC+8下午12:44:54，Roman Shaposhnik写道：
>>
>> On Tue, Sep 18, 2018 at 7:27 PM, Qixuan Wu  wrote:
>> >
>> >
>> > 在 2018年9月19日星期三 UTC+8上午10:00:36，Roman Shaposhnik写道：
>> >>
>> >> On Tue, Sep 18, 2018 at 5:48 PM, Qixuan Wu  wrote:
>> >> >
>> >> >
>> >> > 在 2018年9月19日星期三 UTC+8上午5:56:58，Nadav Har'El写道：
>> >> >>
>> >> >>
>> >> >> On Tue, Sep 18, 2018 at 11:59 AM, Qixuan Wu 
>> >> >> wrote:
>> >> >>>
>> >> >>> Hi all,
>> >> >>>
>> >> >>>We have some requirement to support standard docker OCI image.
>> >> >>>
>> >> >>>Currently OSv did not support 9pfs. Anyone are thinking of this
>> >> >>> or
>> >> >>> not
>> >> >>> ?
>> >> >>
>> >> >>
>> >> >> Supporting 9p and virtfs has been on our wishlist for quite some
>> >> >> time
>> >> >> (see
>> >> >> https://github.com/cloudius-systems/osv/issues/210) but nobody is
>> >> >> actively
>> >> >> working on it.
>> >> >
>> >> >
>> >> > This is a very important feature. If we can do it, it will be easy
>> >> > can
>> >> > support the compatibility with the OCI docker image. Some docker
>> >> > container
>> >> > can securely run inside OSv. That we can call it is another secure
>> >> > container
>> >> > solution, like kata container, gVisor.
>> >>
>> >> FWIW: I was looking to prototype 9pfs support for a hackathon back in
>> >> 2014.
>> >> It looked pretty manageable, frankly. Not sure I can help with coding
>> >> it up again,
>> >> but I can definitely help with whatever else may be needed.
>> >
>> >
>> > I am searching the virtfs/9p source code from linux or some bsd code.
>> > But seems freebsd/netbsd still not support it. I only want the client,
>> > because qemu
>> > has the server already.
>> >
>> > Do you have any some suggestion for the code from where to port ?
>>
>> If what I think you're trying to do is, indeed, what you are actually
>> trying
>> to do, I'd look here:  https://lwn.net/Articles/716582/
>>
>> Thanks,
>> Roman.
>>
>> P.S. Stefano is a good friend of mine -- so if that content seems useful
>> to you I can do a direct intro.
>
>
> I think I will make the NFS (and + vsock later) as the rootfs now. Because
> 9p protocal and the code is too old, and maybe nobody want to maintain it.
> NFS is very popular now.

I tend to disagree. This is how we ended up implementing Docker support
for Xen (which I think is very close to your usecase for OSv -- at
least conceptually):
  
https://github.com/rkt/stage1-xen/blob/master/build/fedora/RUNNING_STAGE1_XEN.md

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About OSv support for docker image using 9pfs

[Roman Shaposhnik]

On Tue, Sep 18, 2018 at 7:27 PM, Qixuan Wu  wrote:
>
>
> 在 2018年9月19日星期三 UTC+8上午10:00:36，Roman Shaposhnik写道：
>>
>> On Tue, Sep 18, 2018 at 5:48 PM, Qixuan Wu  wrote:
>> >
>> >
>> > 在 2018年9月19日星期三 UTC+8上午5:56:58，Nadav Har'El写道：
>> >>
>> >>
>> >> On Tue, Sep 18, 2018 at 11:59 AM, Qixuan Wu  wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>>We have some requirement to support standard docker OCI image.
>> >>>
>> >>>Currently OSv did not support 9pfs. Anyone are thinking of this or
>> >>> not
>> >>> ?
>> >>
>> >>
>> >> Supporting 9p and virtfs has been on our wishlist for quite some time
>> >> (see
>> >> https://github.com/cloudius-systems/osv/issues/210) but nobody is
>> >> actively
>> >> working on it.
>> >
>> >
>> > This is a very important feature. If we can do it, it will be easy can
>> > support the compatibility with the OCI docker image. Some docker
>> > container
>> > can securely run inside OSv. That we can call it is another secure
>> > container
>> > solution, like kata container, gVisor.
>>
>> FWIW: I was looking to prototype 9pfs support for a hackathon back in
>> 2014.
>> It looked pretty manageable, frankly. Not sure I can help with coding
>> it up again,
>> but I can definitely help with whatever else may be needed.
>
>
> I am searching the virtfs/9p source code from linux or some bsd code.
> But seems freebsd/netbsd still not support it. I only want the client,
> because qemu
> has the server already.
>
> Do you have any some suggestion for the code from where to port ?

If what I think you're trying to do is, indeed, what you are actually trying
to do, I'd look here:  https://lwn.net/Articles/716582/

Thanks,
Roman.

P.S. Stefano is a good friend of mine -- so if that content seems useful
to you I can do a direct intro.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About OSv support for docker image using 9pfs

[Roman Shaposhnik]

On Tue, Sep 18, 2018 at 5:48 PM, Qixuan Wu  wrote:
>
>
> 在 2018年9月19日星期三 UTC+8上午5:56:58，Nadav Har'El写道：
>>
>>
>> On Tue, Sep 18, 2018 at 11:59 AM, Qixuan Wu  wrote:
>>>
>>> Hi all,
>>>
>>>We have some requirement to support standard docker OCI image.
>>>
>>>Currently OSv did not support 9pfs. Anyone are thinking of this or not
>>> ?
>>
>>
>> Supporting 9p and virtfs has been on our wishlist for quite some time (see
>> https://github.com/cloudius-systems/osv/issues/210) but nobody is actively
>> working on it.
>
>
> This is a very important feature. If we can do it, it will be easy can
> support the compatibility with the OCI docker image. Some docker container
> can securely run inside OSv. That we can call it is another secure container
> solution, like kata container, gVisor.

FWIW: I was looking to prototype 9pfs support for a hackathon back in 2014.
It looked pretty manageable, frankly. Not sure I can help with coding
it up again,
but I can definitely help with whatever else may be needed.

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Twitter handle

[Roman Shaposhnik]

Are we there yet? ;-) Still getting "Sorry, that page doesn’t exist!"

On Tue, Aug 28, 2018 at 8:38 AM, Waldek Kozaczuk  wrote:
> Funny story! I have just entered OSv birth date (12/1/2012 per 1st git
> commit) as Twitter suggested. And the account got locked because OSv is
> under 13 years old ;-)
> So do not fret if somehow OSv disappeared - I am in process of unlocking the
> account with Twitter.
>
>
> On Tuesday, August 28, 2018 at 8:18:12 AM UTC-4, Waldek Kozaczuk wrote:
>>
>> Recently I have created OSv twitter handle @OSv_unikernel. Please excuse
>> my poor graphics skills - I simply tried to cut it from what was on OSv.io.
>> I hope I did not violate any copyrights.
>>
>> Feel free to follow and spread the news. I will try to be posting
>> regularly about OSv development. If you want something to be posted please
>> let me know.
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: LinuxKit from Docker

[Roman Shaposhnik]

On Fri, Sep 8, 2017 at 7:58 AM, Waldek Kozaczuk  wrote:
> Nadav,
>
> Thanks for your reply. My post was supposed to be thought provoking :-)
>
> Please find my specific comments below.
>
> Regards,
> Waldek
>
> On Thursday, September 7, 2017 at 5:10:59 PM UTC-4, Nadav Har'El wrote:
>>
>>
>> On Thu, Sep 7, 2017 at 8:50 PM, Waldek Kozaczuk 
>> wrote:
>>>
>>> Is linuxkit from Docker (https://github.com/linuxkit/linuxkit) going to
>>> kill unikernels altogether? What do you think?
>>
>>
>> Hi,
>>
>> This approach can definitely work - I have done something like this 17
>> years ago (even before virtualization), when I was working on Linux-based
>> network appliances. We created their images based on custom-compiled Linux
>> kernels (with irrelevant features compiled out) and a tiny hand-picked
>> subset of rpms from the then-common Redhat system.
>
>
> As I understand LinuxKit is simply a tool to create minimalistic Linux
> distributions with enough of Linux (cgroups, chroot, etc) to run docker host
> on it. I was wondering if your understanding is the same as mine.

That is correct. But the kernel is still huge and there's only so much they can
do with kicking unneeded modules and such out of the image. There's been
quite a bit of effort on the embedded side to address the issues of
true modularity
and minimization of Linux kernel but I don't think it is a priority for upstream
maintainers.

With that in mind -- LinuxKit will forever be as big as the Linux
kernel at least.

>> 2. A second question is whether we actually care about it. What kind of
>> application cares if the image size is 10 MB vs 100 MB? I can think of a
>> few, but they are not common.

I can tell you one that clearly does: embedded. And yes virtualization
is very much a thing in embedded nowadays so Unikernels could also be
very much a thing and unlike traditional clouds -- the size will matter
a great deal.

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [RFC] aarch64: basic Xen support

[Roman Shaposhnik]

On Mon, Jan 23, 2017 at 2:51 AM, 'Sergiy Kibrik' via OSv Development
<osv-dev@googlegroups.com> wrote:
> hi Roman,
>
> On 01/20/2017 06:57 PM, Roman Shaposhnik wrote:
>>
>> this is super cool -- I've been very interested in seeing resurrected
>> support for OSv on Xen/ARM
>> for a different project I'm working on. Would love to take a look at your
>> code.
>>
>> Speaking of which -- did you forget to attach a patch or a link to
>> your GH repo? The
>> only link I see is the one to R-Car board
>
>
> I wasn't sure about the better way of patch submission so I've posted all
> patches traditionally to the mailing list, here's thread:
>
> https://groups.google.com/forum/#!topic/osv-dev/15G00wWkB-s
>
> However if github pull request are more suitable -- I will push next
> iteration via pull requests.
> Which way is preferable?

No you're doing everything correctly! The problem is -- all of your PATCH
emails were for some reason filtered out by my email setup -- the only
way I can get to them is to go the Google Groups URL.

But we're all good now. Two questions remains ;-)

1. Is there a GH repo you're working off of? I can recreated the branch
with your patches now that I see them -- but I was wondering if you
could just give me an GH URL (especially if you plan on refactoring based
on the feedback provided so far).

2. Do you plan to also look into Arm32 by any chance? Something along
the lines of Cortex-A7.

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [RFC] aarch64: basic Xen support

[Roman Shaposhnik]

Hi Sergiy!

this is super cool -- I've been very interested in seeing resurrected
support for OSv on Xen/ARM
for a different project I'm working on. Would love to take a look at your code.

Speaking of which -- did you forget to attach a patch or a link to
your GH repo? The
only link I see is the one to R-Car board

Thanks,
Roman.

On Fri, Jan 20, 2017 at 3:04 AM, 'Sergiy Kibrik' via OSv Development
 wrote:
> hi OSv developers,
> Here let me present you with some little work to allow OSv kernel to be loaded
> under Xen hypervisor on AARCH64 platform.
>
> This is basic stuff, which allows to initialise kernel and see log messages.
> No SMP support yet, no heavy PV drivers yet and no application can be run.
>
> Verified on Renesas R-Car board [1] and on qemu-aarch64 (for not to break 
> anything)
>
> [1] http://elinux.org/R-Car/Boards/Salvator-X
>
> Makefile | 5 -
> arch/aarch64/arch-dtb.cc | 9 -
> arch/aarch64/arch-dtb.hh | 7 +++
> arch/aarch64/arch-mmu.hh | 2 +-
> arch/aarch64/arch-setup.cc | 84 
> ++--
> arch/aarch64/boot.S | 3 +++
> arch/aarch64/early-console.cc | 22 --
> arch/aarch64/early-console.hh | 24 ++--
> arch/aarch64/hypercall.S | 30 ++
> arch/aarch64/preboot.S | 17 +
> arch/aarch64/xen.cc | 27 +++
> arch/aarch64/xen.hh | 17 +
> bsd/aarch64/machine/xen/hypercall.h | 73 
> +
> bsd/sys/xen/hypervisor.h | 4 ++--
> bsd/sys/xen/interface/arch-aarch64.h | 52 
> 
> bsd/sys/xen/interface/xen.h | 2 ++
> compiler/include/intrinsics.hh | 2 +-
> drivers/xenconsole.cc | 36 
> drivers/xenconsole.hh | 31 +++
> 19 files changed, 403 insertions(+), 44 deletions(-)
>
> --
> regards,
> Sergiy
>
> --
> You received this message because you are subscribed to the Google Groups 
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to osv-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Problems in sendfile?

[Roman Shaposhnik]

On Sun, Dec 4, 2016 at 12:34 AM, Avi Kivity <a...@scylladb.com> wrote:
> On 12/04/2016 04:00 AM, Roman Shaposhnik wrote:
>>
>> sendfile is a pretty Linux-centric API and I'm sure it isn't implemented
>> by OSv.
>
>
> OSv aims to be Linux compatible, and does implement sendfile() (though not
> without problems, as we see).
>
> sendfile() originated in Solaris, IIRC, and variants exist in the BSDs as
> well.

Good info -- I stand corrected!

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Problems in sendfile?

[Roman Shaposhnik]

sendfile is a pretty Linux-centric API and I'm sure it isn't implemented by OSv.

Given that this is erlang and it must support platforms that don't
have sendfile(2)
I suggest you look into how to build it for POSIX (as opposed to
Linux) environment

Thanks,
Roman.

On Sat, Dec 3, 2016 at 5:06 PM, Rick Payne  wrote:
>
> The erlang app that I’m hacking is sending out some data to a socket and its 
> using sendfile to do this. This is causing OSv to abort:
>
> (gdb) bt
> #0  processor::cli_hlt () at arch/x64/processor.hh:248
> #1  0x00209cf6 in arch::halt_no_interrupts () at arch/x64/arch.hh:48
> #2  0x0049983a in osv::halt () at arch/x64/power.cc:24
> #3  0x0022d3c5 in abort (fmt=0xa1555d "Aborted\n") at runtime.cc:130
> #4  0x0022d290 in abort () at runtime.cc:96
> #5  0x006a49c1 in osv::generate_signal (siginfo=..., 
> ef=0x80007f40e068) at libc/signal.cc:128
> #6  0x006a4aa1 in osv::handle_mmap_fault (addr=35184720220160, sig=7, 
> ef=0x80007f40e068) at libc/signal.cc:139
> #7  0x003c6363 in mmu::vm_sigbus (addr=35184720220160, 
> ef=0x80007f40e068) at core/mmu.cc:1323
> #8  0x003c7715 in mmu::file_vma::fault (this=0xad436c80, 
> addr=35184720220160, ef=0x80007f40e068) at core/mmu.cc:1691
> #9  0x003c6513 in mmu::vm_fault (addr=35184720220160, 
> ef=0x80007f40e068) at core/mmu.cc:1342
> #10 0x004899bb in page_fault (ef=0x80007f40e068) at 
> arch/x64/mmu.cc:38
> #11 
> #12 0x0047e905 in repmovsq (n=, src=, 
> dest=) at arch/x64/string.cc:90
> #13 memcpy_repmov_old_ssse3 (dest=0x88898000, src=0x200014c01000, 
> n=4096) at arch/x64/string.cc:270
> #14 0x00676d29 in uiomove (cp=0x88898000, n=4096, 
> uio=0x205e9c70) at fs/vfs/subr_uio.cc:62
> #15 0x0023d78e in m_uiotombuf (uio=0x205e9c70, how=2, len=33397, 
> align=0, min_size=2048, flags=0)
> at bsd/sys/kern/uipc_mbuf.cc:1847
> #16 0x00245753 in sosend_generic (so=0xa252e200, addr=0x0, 
> uio=0x205e9c70, top=0x0, control=0x0, flags=0, td=0x0)
> at bsd/sys/kern/uipc_socket.cc:1047
> #17 0x00245ece in sosend (so=0xa252e200, addr=0x0, 
> uio=0x205e9c70, top=0x0, control=0x0, flags=0, td=0x0)
> at bsd/sys/kern/uipc_socket.cc:1282
> #18 0x00253d41 in socket_file::write (this=0xad1a1f80, 
> uio=0x205e9c70, flags=0) at bsd/sys/kern/sys_socket.cc:91
> #19 0x0067d3a0 in sys_write (fp=0xad1a1f80, 
> iov=0x205e9d10, niov=1, offset=-1, count=0x205e9d00)
> at fs/vfs/vfs_syscalls.cc:311
> #20 0x0064c09a in pwrite (fd=31, buf=0x200014c0, 
> count=1073741823, offset=-1) at fs/vfs/main.cc:387
> #21 0x0064c12b in write (fd=31, buf=0x200014c0, count=1073741823) 
> at fs/vfs/main.cc:405
> #22 0x0064f003 in sendfile (out_fd=31, in_fd=32, 
> _offset=0x223889e8, count=1073741823) at fs/vfs/main.cc:2051
> #23 0x11616d64 in efile_sendfile (errInfo=0x2238894c, in_fd=32, 
> out_fd=31, offset=0x223889e8, nbytes=0x205e9e78,
> hdtl=0x0) at drivers/unix/unix_efile.c:930
> #24 0x1163527a in invoke_sendfile (data=0x22388918) at 
> drivers/common/efile_drv.c:1917
> #25 0x115a98eb in async_main (arg=0x2098f0c0) at 
> beam/erl_async.c:509
> #26 0x116c0615 in thr_wrapper (vtwd=0x201fe810) at 
> pthread/ethread.c:114
> #27 0x0069eebf in 
> pthread_private::pthread::::operator()(void) const 
> (__closure=0xa0007f39e700)
> at libc/pthread.cc:114
> #28 0x006a1a22 in std::_Function_handler pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const 
> pthread_private::thread_attr*):: >::_M_invoke(const std::_Any_data 
> &) (__functor=...) at /usr/include/c++/5/functional:1871
> #29 0x0044cb1c in std::function::operator()() const 
> (this=0x80007f409070) at /usr/include/c++/5/functional:2267
> #30 0x005bfac4 in sched::thread::main (this=0x80007f409040) at 
> core/sched.cc:1171
> #31 0x005bbca6 in sched::thread_main_c (t=0x80007f409040) at 
> arch/x64/arch-switch.hh:164
> #32 0x00489793 in thread_main () at arch/x64/entry.S:113
>
> It looks like the erlang code is calling sendfile with a large value (larger 
> than the size of the actual file being sent). I see the sendfile code 
> calculating the bytes to map, then mapping that area. It then crashes during 
> the write:
>
> 930 retval = sendfile(out_fd, in_fd, offset, SENDFILE_CHUNK_SIZE);
> #22 0x0064f003 in sendfile (out_fd=31, in_fd=32, 
> _offset=0x223889e8, count=1073741823) at fs/vfs/main.cc:2051
> 2051auto ret = write(out_fd, src + (offset % PAGESIZE), count);
> (gdb) p/x count
> $20 = 0x3fff
>
> Any ideas?
>
> Cheers,
> Rick
>
> --
> You received this message because you are subscribed to the Google Groups 
> "OSv 

Re: ScyllaDB on OSv?

[Roman Shaposhnik]

On Fri, Aug 12, 2016 at 10:33 AM, Avi Kivity  wrote:
> On 08/12/2016 08:23 PM, Waldek Kozaczuk wrote:
>>
>> Given that ScyllaDB uses seastar that can run on OSv is it possible to run
>> ScyllaDB on OSv? I have not found anything on scyllaDB site ?
>>
>> If no are there any plans do make it possible? If yes would there be any
>> advantage of running scylladb on top of OSV performance-wise?
>
>
> Right now OSv lacks a good aio-enabled filesystem.  ZFS is designed to
> provide clever caching and file management tools (ARC, snapshots) while
> ScyllaDB wants to do its own caching and DMA file data directly from disk
> using AIO.

What's your recommended FS on Linux?

Thanks,
Roman.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.