I thing I forgot to mention is that currently OSv arm version only supports
GIC 2 interrupt controller which is enough for RP 4. But we need GIC 3/4 to
be able to run on Firecracker on "real"/non-toy hardware. I do not know how
difficult it would be to support GIC 3/4.
On Friday, May 22, 2020 at 12:21:23 AM UTC-4, Roman Shaposhnik wrote:
>
> This is super awesome, Waldek! I'm now inspired to run OSv as a VM
> backed by Firecracker on Project EVE/RPi! ;-)
>
> Btw, can you share your osv-config.json and also details of what
> version of Firecracker you used?
>
{
"boot-source": {
"kernel_image_path": "/home/ubuntu/loader_with_console.img",
"boot_args": "console=ttyS0 --verbose --nomount --maxnic=0
/tools/hello.so"
},
"drives": []
}
firecracker-v0.21.1-aarch64 - which I believe is the latest.
> Better yet -- if there's a github repo with all the bits and pieces
> required to replicate your setup -- that'd be worthy of a gift of a
> beer (post COVID tho!) ;-)
>
There is a build-able branch (which is in essence current master + patch I
have enclosed) - https://github.com/wkozaczuk/osv/tree/arm_on_firecracker2
./scripts/build image=empty arch=aarch64 fs=ramfs (it add simple hello
world binary).
and then run
./firecracker-v0.21.1-aarch64 --no-api --config-file ./osv-config.json
>
> Thanks,
> Roman.
>
> On Thu, May 21, 2020 at 1:58 PM Waldek Kozaczuk <jwkoz...@gmail.com
> <javascript:>> wrote:
> >
> >
> > ubuntu@ubuntu-rasp4:~$ uname -a
> >
> > Linux ubuntu-rasp4 5.3.0-1025-raspi2 #27-Ubuntu SMP Fri May 8 08:32:04
> UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
> >
> > ubuntu@ubuntu-rasp4:~$ ./firecracker-v0.21.1-aarch64 --no-api
> --config-file ./osv-config.json
> >
> > OSv v0.55.0-9-g840428ad
> >
> > PSCI: version 65536.0 detected.
> >
> > setup_arm_clock() ENTERED, lr=00000000800d54e0
> >
> > arm_clock(): frequency read as 000000000337f980
> >
> > interrupt_table::interrupt_table() ENTERED, lr=00000000800a8d1c
> >
> > gic_driver::init_cpu() ENTERED, lr=0000000080201c80
> >
> > CPU interface enabled.
> >
> > gic_driver::init_dist() ENTERED, lr=0000000080201c8c
> >
> > number of supported IRQs: 0000000000000080
> >
> > interrupt table: gic driver created.
> >
> > registered IRQ id=0000000000000004
> >
> > registered IRQ id=000000000000001b
> >
> > registered IRQ id=0000000000000000
> >
> > Premain complete!
> >
> > smp_launch ENTERED, lr=00000000800d709c
> >
> > Booted up in 0.00 ms
> >
> > Cmdline: console=ttyS0 --verbose --nomount --maxnic=0 /tools/hello.so
> earlycon=uart,mmio,0x40001000
> >
> > faulting address 000010000001fea0
> >
> > faulting address 0000100000000328
> >
> > faulting address 0000100000020000
> >
> > faulting address 0000100000050018
> >
> > faulting address 00001000000301c8
> >
> > faulting address 000010000004fe00
> >
> > Hello from C code
> >
> > 2020-05-21T20:24:04.080765641 [anonymous-instance:ERROR:src/vmm/src/
> vstate.rs:951] Unexpected exit reason on vcpu run: SystemEvent
> >
> >
> > In general it took me a bit of research as I am not really familiar with
> ARM architecture and even reading the assembly was a bit a challenge to say
> the least. And then debugging without debugger and any console (:-( .. so
> not debug() for long time. But all in all it was not too bad and the
> changes that I had to make to OSv are in my opinion much smaller and easier
> comparing to x86_64.
> >
> > Below you will see the "hack-patch" showing what changes I had to make.
> Logically, following things had to be changed:
> >
> > The most important thing was to move kernel from 0x40000000 (1GB) to
> 0x80000000 (2GB) which required changing one line in Makefile (see below)
> and changing boot paging table to map the 2GB-3GB area of memory; only then
> I could actually start debugging :-) I wonder if it will also work on
> QEMU/KVM - possinly qemu boot loader will inspect ELF and place it
> accordingly in memory; firecracker does not read ELF and simply places it
> at 2GB
> > To get console working I guessed that that need to create equivalent
> class for isa_serial_console but communicating over mmio (see
> mmio_isa_serial_console.hh/cc) which in essence invokes mmio_set*/mmio_get*
> (it would be nice to extract common code somehow - suggestions welcome)
> > Some more trivial changes - for now mostly disabling things - for now
> quite disorganized
> >
> >
> > diff --git a/Makefile b/Makefile
> > index db3c68cf..ffd570a5 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -354,6 +354,7 @@ tools := tools/mkfs/mkfs.so tools/cpiod/cpiod.so
> > $(out)/tools/%.o: COMMON += -fPIC
> >
> > tools += tools/uush/uush.so
> > +tools += tools/uush/hello.so
> > tools += tools/uush/ls.so
> > tools += tools/uush/mkdir.so
> >
> > @@ -451,8 +452,8 @@ endif # x64
> >
> > ifeq ($(arch),aarch64)
> >
> > -kernel_base := 0x40080000
> > -kernel_vm_base := 0x40080000
> > +kernel_base := 0x80080000
> > +kernel_vm_base := 0x80080000
> > app_local_exec_tls_size := 0x0
> >
> > include $(libfdt_base)/Makefile.libfdt
> > @@ -816,6 +817,7 @@ drivers += drivers/xenplatform-pci.o
> > endif # x64
> >
> > ifeq ($(arch),aarch64)
> > +drivers += drivers/mmio-isa-serial.o
> > drivers += drivers/pl011.o
> > drivers += drivers/xenconsole.o
> > drivers += drivers/virtio.o
> > diff --git a/arch/aarch64/arch-dtb.cc b/arch/aarch64/arch-dtb.cc
> > index b59f1dcc..cd0719e8 100644
> > --- a/arch/aarch64/arch-dtb.cc
> > +++ b/arch/aarch64/arch-dtb.cc
> > @@ -225,7 +225,8 @@ bool dtb_get_gic_v2(u64 *dist, size_t *dist_len, u64
> *cpu, size_t *cpu_len)
> > if (!dtb)
> > return false;
> >
> > - node = fdt_node_offset_by_compatible(dtb, -1,
> "arm,cortex-a15-gic");
> > + //node = fdt_node_offset_by_compatible(dtb, -1,
> "arm,cortex-a15-gic");
> > + node = fdt_node_offset_by_compatible(dtb, -1, "arm,gic-400");
> > if (node < 0)
> > return false;
> >
> > @@ -613,11 +614,13 @@ void __attribute__((constructor(init_prio::dtb)))
> dtb_setup()
> > abort("dtb_setup: failed to parse cpu mpid.\n");
> > }
> >
> > + //TODO: Parse PCI only if required and
> > + // abort if missing only if required
> > dtb_timer_irq = dtb_parse_timer_irq();
> > dtb_pci_irqmask = dtb_parse_pci_irqmask();
> > dtb_pci_irqmap_count = dtb_parse_pci_irqmap_count();
> > if (!dtb_parse_pci_irqmap(dtb_pci_bdfs, dtb_pci_irq_ids,
> dtb_pci_irqmap_count)) {
> > - abort("dtb_setup: failed to parse pci_irq_map.\n");
> > + //abort("dtb_setup: failed to parse pci_irq_map.\n");
> > }
> >
> > register u64 edata;
> > diff --git a/arch/aarch64/arch-setup.cc b/arch/aarch64/arch-setup.cc
> > index bdbb1266..b12546ae 100644
> > --- a/arch/aarch64/arch-setup.cc
> > +++ b/arch/aarch64/arch-setup.cc
> > @@ -17,6 +17,7 @@
> > #include <osv/debug.hh>
> > #include <osv/commands.hh>
> > #include <osv/xen.hh>
> > +#include <osv/version.h>
> >
> > #include "arch-mmu.hh"
> > #include "arch-dtb.hh"
> > @@ -25,6 +26,7 @@
> > #include "drivers/pl011.hh"
> > #include "early-console.hh"
> > #include <osv/pci.hh>
> > +#include "drivers/mmio-isa-serial.hh"
> >
> > #include <alloca.h>
> >
> > @@ -93,9 +95,10 @@ void arch_setup_free_memory()
> >
> > if (!is_xen()) {
> > /* linear_map [TTBR0 - UART] */
> > - addr =
> (mmu::phys)console::aarch64_console.pl011.get_base_addr();
> > - mmu::linear_map((void *)addr, addr, 0x1000, mmu::page_size,
> > - mmu::mattr::dev);
> > + //TODO: Only run it if pl011 is detected
> > + //addr =
> (mmu::phys)console::aarch64_console.pl011.get_base_addr();
> > + //mmu::linear_map((void *)addr, addr, 0x1000, mmu::page_size,
> > + // mmu::mattr::dev);
> > }
> >
> > /* linear_map [TTBR0 - GIC DIST and GIC CPU] */
> > @@ -110,12 +113,17 @@ void arch_setup_free_memory()
> > mmu::linear_map((void *)cpu, (mmu::phys)cpu, cpu_len,
> mmu::page_size,
> > mmu::mattr::dev);
> >
> > - arch_setup_pci();
> > + //TODO: Call it only if PCI available
> > + //arch_setup_pci();
> >
> > // get rid of the command line, before memory is unmapped
> > osv::parse_cmdline(cmdline);
> >
> > mmu::switch_to_runtime_page_tables();
> > +
> > + //TODO: This is probably wrong place to init the console
> > + arch_init_early_console();
> > + debug_early("OSv " OSV_VERSION "\n");
> > }
> >
> > void arch_setup_tls(void *tls, const elf::tls_data& info)
> > @@ -185,27 +193,35 @@ void arch_init_early_console()
> > return;
> > }
> >
> > + /* Comment out for now
> > new (&console::aarch64_console.pl011) console::PL011_Console();
> > console::arch_early_console = console::aarch64_console.pl011;
> > int irqid;
> > - u64 addr = dtb_get_uart(&irqid);
> > - if (!addr) {
> > + u64 addr = dtb_get_uart(&irqid);*/
> > + //if (!addr) {
> > /* keep using default addresses */
> > - return;
> > - }
> > + // return;
> > + //}
> >
> > - console::aarch64_console.pl011.set_base_addr(addr);
> > - console::aarch64_console.pl011.set_irqid(irqid);
> > + //console::aarch64_console.pl011.set_base_addr(addr);
> > + //console::aarch64_console.pl011.set_irqid(irqid);
> > +
> > + new (&console::aarch64_console.mmio_isa_serial)
> console::mmio_isa_serial_console();
> > + console::arch_early_console =
> console::aarch64_console.mmio_isa_serial;
> > +
> > + console::mmio_isa_serial_console::early_init();
> > }
> >
> > bool arch_setup_console(std::string opt_console)
> > {
> > + /*
> > if (opt_console.compare("pl011") == 0) {
> > console::console_driver_add(&console::arch_early_console);
> > } else if (opt_console.compare("all") == 0) {
> > console::console_driver_add(&console::arch_early_console);
> > } else {
> > return false;
> > - }
> > + }*/
> > + console::console_driver_add(&console::arch_early_console);
> > return true;
> > }
> > diff --git a/arch/aarch64/boot.S b/arch/aarch64/boot.S
> > index b29fd16a..1d2af75c 100644
> > --- a/arch/aarch64/boot.S
> > +++ b/arch/aarch64/boot.S
> > @@ -188,21 +188,15 @@ ident_pt_l4_ttbr0:
> > .quad 0
> > .endr
> > ident_pt_l3_ttbr0:
> > - .quad ident_pt_l2_0_ttbr0 + 0x3
> > - .quad ident_pt_l2_1_ttbr0 + 0x3
> > - .rept 510
> > + .quad 0
> > + .quad 0
> > + .quad ident_pt_l2_2_ttbr0 + 0x3 // Map 4GB-6GB one-to-one
> > + .rept 509
> > .quad 0
> > .endr
> > -ident_pt_l2_0_ttbr0:
> > - index = 0
> > - offset = 0x00000000
> > - .rept 512
> > - .quad (index << 21) + 0x401
> > - index = index + 1
> > - .endr
> > -ident_pt_l2_1_ttbr0:
> > +ident_pt_l2_2_ttbr0:
> > index = 0
> > - offset = 0x40000000
> > + offset = 0x80000000
> > .rept 512
> > .quad offset + (index << 21) + 0x411
> > index = index + 1
> > diff --git a/arch/aarch64/early-console.cc
> b/arch/aarch64/early-console.cc
> > index d5196c74..8c91c965 100644
> > --- a/arch/aarch64/early-console.cc
> > +++ b/arch/aarch64/early-console.cc
> > @@ -13,6 +13,7 @@
> > namespace console {
> >
> > union AARCH64_Console aarch64_console;
> > -console_driver & arch_early_console = aarch64_console.pl011;
> > +//console_driver & arch_early_console = aarch64_console.pl011;
> > +console_driver & arch_early_console = aarch64_console.mmio_isa_serial;
> >
> > }
> > diff --git a/arch/aarch64/early-console.hh
> b/arch/aarch64/early-console.hh
> > index cbc2cac4..cd39738b 100644
> > --- a/arch/aarch64/early-console.hh
> > +++ b/arch/aarch64/early-console.hh
> > @@ -11,12 +11,15 @@
> > #include <drivers/console-driver.hh>
> > #include <drivers/pl011.hh>
> > #include <drivers/xenconsole.hh>
> > +#include <drivers/mmio-isa-serial.hh>
> > +
> >
> > namespace console {
> >
> > union AARCH64_Console {
> > PL011_Console pl011;
> > XEN_Console xen;
> > + mmio_isa_serial_console mmio_isa_serial;
> >
> > AARCH64_Console() {}; /* placement new is used to initialize
> object */
> > ~AARCH64_Console() {}; /* won't ever be called */
> > diff --git a/arch/aarch64/mmu.cc b/arch/aarch64/mmu.cc
> > index fef48e5a..012d3c57 100644
> > --- a/arch/aarch64/mmu.cc
> > +++ b/arch/aarch64/mmu.cc
> > @@ -18,11 +18,11 @@ void page_fault(exception_frame *ef)
> > {
> > sched::fpu_lock fpu;
> > SCOPE_LOCK(fpu);
> > - debug_early_entry("page_fault");
> > + //debug_early_entry("page_fault");
> > u64 addr;
> > asm volatile ("mrs %0, far_el1" : "=r"(addr));
> > debug_early_u64("faulting address ", (u64)addr);
> > - debug_early_u64("elr exception ra ", (u64)ef->elr);
> > + //debug_early_u64("elr exception ra ", (u64)ef->elr);
> >
> > if (fixup_fault(ef)) {
> > debug_early("fixed up with fixup_fault\n");
> > @@ -45,7 +45,7 @@ void page_fault(exception_frame *ef)
> > mmu::vm_fault(addr, ef);
> > }
> >
> > - debug_early("leaving page_fault()\n");
> > + //debug_early("leaving page_fault()\n");
> > }
> >
> > namespace mmu {
> > diff --git a/drivers/mmio-isa-serial.cc b/drivers/mmio-isa-serial.cc
> > new file mode 100644
> > index 00000000..c59cec8d
> > --- /dev/null
> > +++ b/drivers/mmio-isa-serial.cc
> > @@ -0,0 +1,134 @@
> > +/*
> > + * Copyright (C) 2013 Cloudius Systems, Ltd.
> > + *
> > + * This work is open source software, licensed under the terms of the
> > + * BSD license as described in the LICENSE file in the top-level
> directory.
> > + */
> > +
> > +#include "mmio-isa-serial.hh"
> > +
> > +namespace console {
> > +
> > +// UART registers, offsets to ioport:
> > +enum regs {
> > + IER = 1, // Interrupt Enable Register
> > + FCR = 2, // FIFO Control Register
> > + LCR = 3, // Line Control Register
> > + MCR = 4, // Modem Control Register
> > + LSR = 5, // Line Control Register
> > + MSR = 6, // Modem Status Register
> > + SCR = 7, // Scratch Register
> > + DLL = 0, // Divisor Latch LSB Register
> > + DLM = 1, // Divisor Latch MSB Register
> > +};
> > +
> > +enum lcr {
> > + // When bit 7 (DLAB) of LCR is set to 1, the two registers 0 and 1
> > + // change their meaning and become two bytes controlling the baud
> rate
> > + DLAB = 0x80, // Divisor Latch Access Bit in LCR register
> > + LEN_8BIT = 3,
> > +};
> > +
> > +// Various bits of the Line Status Register
> > +enum lsr {
> > + RECEIVE_DATA_READY = 0x1,
> > + OVERRUN = 0x2,
> > + PARITY_ERROR = 0x4,
> > + FRAME_ERROR = 0x8,
> > + BREAK_INTERRUPT = 0x10,
> > + TRANSMIT_HOLD_EMPTY = 0x20,
> > + TRANSMIT_EMPTY = 0x40,
> > + FIFO_ERROR = 0x80,
> > +};
> > +
> > +// Various bits of the Modem Control Register
> > +enum mcr {
> > + DTR = 0x1,
> > + RTS = 0x2,
> > + AUX_OUTPUT_1 = 0x4,
> > + AUX_OUTPUT_2 = 0x8,
> > + LOOPBACK_MODE = 0x16,
> > +};
> > +
> > +mmioaddr_t mmio_isa_serial_console::_addr_mmio;
> > +
> > +void mmio_isa_serial_console::early_init()
> > +{
> > + u64 address = 0x40001000; //TODO: Should parse from boot command
> line ('earlycon=uart,mmio,0x40001000')
> > + u64 size = 4096;
> > +
> > + _addr_mmio = mmio_map(address, size);
> > +
> > + // Set the UART speed to to 115,200 bps, This is done by writing
> 1,0 to
> > + // Divisor Latch registers, but to access these we need to
> temporarily
> > + // set the Divisor Latch Access Bit (DLAB) on the LSR register,
> because
> > + // the UART has fewer ports than registers...
> > + mmio_setb(_addr_mmio + (int)regs::LCR, lcr::LEN_8BIT | lcr::DLAB);
> > + mmio_setb(_addr_mmio + (int)regs::DLL, 1);
> > + mmio_setb(_addr_mmio + (int)regs::DLM, 0);
> > + mmio_setb(_addr_mmio + (int)regs::LCR, lcr::LEN_8BIT);
> > +
> > + // interrupt threshold
> > + mmio_setb(_addr_mmio + (int)regs::FCR, 0);
> > +
> > + // disable interrupts
> > + mmio_setb(_addr_mmio + (int)regs::IER, 0);
> > +
> > + // Most physical UARTs need the MCR AUX_OUTPUT_2 bit set to 1 for
> > + // interrupts to be generated. QEMU doesn't bother checking this
> > + // bit, but interestingly VMWare does, so we must set it.
> > + mmio_setb(_addr_mmio + (int)regs::MCR, mcr::AUX_OUTPUT_2);
> > +}
> > +
> > +void mmio_isa_serial_console::write(const char *str, size_t len)
> > +{
> > + while (len-- > 0)
> > + putchar(*str++);
> > +}
> > +
> > +bool mmio_isa_serial_console::input_ready()
> > +{
> > + u8 val = mmio_getb(_addr_mmio + (int)regs::LSR);
> > + // On VMWare hosts without a serial port, this register always
> > + // returns 0xff. Just ignore it instead of spinning incessantly.
> > + return (val != 0xff && (val & lsr::RECEIVE_DATA_READY));
> > +}
> > +
> > +char mmio_isa_serial_console::readch()
> > +{
> > + u8 val;
> > + char letter;
> > +
> > + do {
> > + val = mmio_getb(_addr_mmio + (int)regs::LSR);
> > + } while (!(val & (lsr::RECEIVE_DATA_READY | lsr::OVERRUN |
> lsr::PARITY_ERROR | lsr::FRAME_ERROR)));
> > +
> > + letter = mmio_getb(_addr_mmio);
> > +
> > + return letter;
> > +}
> > +
> > +void mmio_isa_serial_console::putchar(const char ch)
> > +{
> > + u8 val;
> > +
> > + do {
> > + val = mmio_getb(_addr_mmio + (int)regs::LSR);
> > + } while (!(val & lsr::TRANSMIT_HOLD_EMPTY));
> > +
> > + mmio_setb(_addr_mmio, ch);
> > +}
> > +
> > +void mmio_isa_serial_console::enable_interrupt()
> > +{
> > + // enable interrupts
> > + mmio_setb(_addr_mmio + (int)regs::IER, 1);
> > +}
> > +
> > +void mmio_isa_serial_console::dev_start() {
> > + //TODO: Figure out which interrupt and what kind to use
> > + //_irq.reset(new sgi_edge_interrupt(4, [&] { _thread->wake(); }));
> > + enable_interrupt();
> > +}
> > +
> > +}
> > diff --git a/drivers/mmio-isa-serial.hh b/drivers/mmio-isa-serial.hh
> > new file mode 100644
> > index 00000000..f422dfda
> > --- /dev/null
> > +++ b/drivers/mmio-isa-serial.hh
> > @@ -0,0 +1,39 @@
> > +/*
> > + * Copyright (C) 2013 Cloudius Systems, Ltd.
> > + *
> > + * This work is open source software, licensed under the terms of the
> > + * BSD license as described in the LICENSE file in the top-level
> directory.
> > + */
> > +
> > +#ifndef DRIVERS_MMIO_ISA_SERIAL_HH
> > +#define DRIVERS_MMIO_ISA_SERIAL_HH
> > +
> > +#include "console-driver.hh"
> > +#include <osv/pci.hh>
> > +#include <osv/sched.hh>
> > +#include <osv/interrupt.hh>
> > +#include <osv/mmio.hh>
> > +
> > +namespace console {
> > +
> > +class mmio_isa_serial_console : public console_driver {
> > +public:
> > + static void early_init();
> > + virtual void write(const char *str, size_t len);
> > + virtual void flush() {}
> > + virtual bool input_ready() override;
> > + virtual char readch();
> > +
> > +private:
> > + //std::unique_ptr<sgi_edge_interrupt> _irq;
> > + static mmioaddr_t _addr_mmio;
> > +
> > + virtual void dev_start();
> > + void enable_interrupt();
> > + static void putchar(const char ch);
> > + virtual const char *thread_name() { return "mmio-isa-serial-input";
> }
> > +};
> > +
> > +}
> > +
> > +#endif
> > diff --git a/loader.cc b/loader.cc
> > index 66bfb52c..9867ca49 100644
> > --- a/loader.cc
> > +++ b/loader.cc
> > @@ -94,12 +94,12 @@ extern "C" {
> >
> > void premain()
> > {
> > - arch_init_early_console();
> > + //arch_init_early_console();
> >
> > /* besides reporting the OSV version, this string has the function
> > to check if the early console really works early enough,
> > without depending on prior initialization. */
> > - debug_early("OSv " OSV_VERSION "\n");
> > + //debug_early("OSv " OSV_VERSION "\n");
> >
> > arch_init_premain();
> >
> > @@ -116,6 +116,7 @@ void premain()
> > for (auto init = inittab.start; init < inittab.start +
> inittab.count; ++init) {
> > (*init)();
> > }
> > + debug_early("Premain complete!\n");
> > boot_time.event(".init functions");
> > }
> >
> > @@ -230,6 +231,7 @@ static void parse_options(int loader_argc, char**
> loader_argv)
> > opt_maxnic = true;
> > maxnic = options::extract_option_int_value(options_values,
> "maxnic", handle_parse_error);
> > }
> > + maxnic = 0;
> >
> > if (extract_option_flag(options_values, "trace-backtrace")) {
> > opt_log_backtrace = true;
> > @@ -261,6 +263,7 @@ static void parse_options(int loader_argc, char**
> loader_argv)
> > }
> >
> > opt_mount = !extract_option_flag(options_values, "nomount");
> > + opt_mount = false;
> > opt_pivot = !extract_option_flag(options_values, "nopivot");
> > opt_random = !extract_option_flag(options_values, "norandom");
> > opt_init = !extract_option_flag(options_values, "noinit");
> > @@ -351,7 +354,8 @@ std::vector<std::vector<std::string> >
> prepare_commands(char* app_cmdline)
> > bool ok;
> >
> > printf("Cmdline: %s\n", app_cmdline);
> > - commands = osv::parse_command_line(app_cmdline, ok);
> > + //commands = osv::parse_command_line(app_cmdline, ok);
> > + commands = osv::parse_command_line("/tools/hello.so", ok);
> >
> > if (!ok) {
> > puts("Failed to parse command line.");
> > diff --git a/usr_ramfs.manifest.skel b/usr_ramfs.manifest.skel
> > index 4bdfdd32..1a04c22f 100644
> > --- a/usr_ramfs.manifest.skel
> > +++ b/usr_ramfs.manifest.skel
> > @@ -3,6 +3,8 @@
> > /libvdso.so: libvdso.so
> > /tools/mount-fs.so: tools/mount/mount-fs.so
> > /tools/umount.so: tools/mount/umount.so
> > +/tools/uush.so: tools/uush/uush.so
> > +/tools/hello.so: tools/uush/hello.so
> > /usr/lib/libgcc_s.so.1: %(libgcc_s_dir)s/libgcc_s.so.1
> > /&/etc/hosts: ../../static/&
> >
> >
> >
> > Please not that I have not tested how it works with multiple CPUs or if
> virtual block and network devices operate.
> >
> > Waldek
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "OSv Development" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to osv...@googlegroups.com <javascript:>.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/f4c6cafe-690a-4619-b7b6-e414e3f9343a%40googlegroups.com.
>
>
>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/ea85f3d5-4ddb-4d8a-943e-95e4b2459300%40googlegroups.com.
