Re: [osv-dev] Next releases - 0.56 and onward - proposal

2020-06-07 Thread Roman Shaposhnik
This is such an impressive list! Would love to see it all!

Thanks,
Roman.

On Sat, Jun 6, 2020 at 9:45 AM Waldek Kozaczuk  wrote:
>
> Hi,
>
> Here is a list of things, not necessarily in any importance order, I am 
> proposing we try to accomplish for next releases:
>
> Prune the list of issues
>
> Close anything that is irrelevant or possibly completed
>
> One subcategory I can think of, are issues about improving java wrapper like 
> this one - https://github.com/cloudius-systems/osv/issues/169. Given we can 
> run unmodified /usr/bin/java without need of a wrapper and given the wrapper 
> (java.so) does not support (or is even compatible) with many features of Java 
> 9 and higher, I suggest we close any issues like 169 which are about 
> improving java.so - I do not think it is worth any time investment (the only 
> real benefit of java.so is ballooning feature which is broken - 
> https://github.com/cloudius-systems/osv/issues/796 and 
> https://github.com/cloudius-systems/osv/issues/1038 [duplicate of 796?] - but 
> if anyone is interested in fixing it I think we might support it without 
> java.so anyway).
>
> Better annotate existing issues and especially select ones as '"good first 
> issue" (see 
> https://github.blog/2020-01-22-browse-good-first-issues-to-start-contributing-to-open-source/)
>
> Better support Linux executables built for Musl-based distributions like 
> Alpine (see https://wiki.musl-libc.org/projects-using-musl.html)
>
> These may work even better given OSv libc implementation is to great extent 
> based on musl
>
> Tooling
>
> Improve capstan
>
> Better support QEMU on OSX (new accelerators)
> Support pulling artifacts from GitHub nightly releases repo
> Support running Firecracker, QEMU microvm
> Support Virtio-FS
>
> Create plugins to deploy Java, Node, etc apps
>
> IntelliJ
> Gradle
> Maven
>
> Run Docker images (possibly musl-based like Alpine are better)
>
> There is already existing example - 
> https://github.com/cloudius-systems/osv-apps/tree/master/openjdk12-jre-from-docker
>
> Review the lazy app thread stack patch aimed to reduce memory consumption
>
> https://groups.google.com/d/msg/osv-dev/tZnwiScmjZY/GkY0hV9EAwAJ
>
> Modularization/"Librarization"
>
> See the discussion - 
> https://groups.google.com/d/msg/osv-dev/BHwN6Stm3n4/arvdOwl3AAAJ
> Hide most symbols in the kernel
>
> Issue #97 - Be more selective on symbols exported from the kernel
> See related discussion - 
> https://groups.google.com/d/msg/osv-dev/fs-jmyaXY_E/vWUk-wDDAAAJ
>
> Reduce kernel size:
>
> See the discussion - 
> https://groups.google.com/d/msg/osv-dev/hCYGRzytaJ4/D23S_ibNAgAJ and 
> https://groups.google.com/d/msg/osv-dev/JsJfnhTX_EY/PKUvYm9-AQAJ
>
> Expose C++ APIs as C to hide C++ completely
>
> Keep improving Virtio-FS (#1079)
>
> Integrate with pagecache
> Improve DAX
>
> Improve ARM support
>
> See this Firecracker patch - 
> https://groups.google.com/d/msg/osv-dev/3TrFxpW9Bhk/AdilzZT5AQAJ
> Improve run.py
>
> Reach out to Spirent to see if any more networking related bugs fixed can be 
> brought over from the https://github.com/SpirentOrion/osv fork
> Add support of Virtio GPU/VGA
> Upgrade musl
>
> I kind of remember Geraldo Netto started on this
>
> Test newer runtimes
>
> Golang >= 1.13
> Java > 12
> DENO 1.0
>
> Support .NET Core executables
>
> See what needs to be finished in this thread - 
> https://groups.google.com/d/msg/osv-dev/k69cHw7qvTg/9NqPq70IAwAJ
>
> Refresh look and feel of http://osv.io
>
> Other interesting existing issues:
>
> Better deal with small amount of memory - 
> https://github.com/cloudius-systems/osv/issues/1013
> #28 - Optimize atomic operations on UP (single-VCPU)
> #1016 - Add support for smb2/3 (samba) file system T
>
> There is already a patch sent to the list - 
> https://groups.google.com/d/msg/osv-dev/QToC22-QHiA/ejSOMGe7BwAJ
>
> #1009 - Make ZFS optional as a shared library
> #212 - Support static executables
> #1069 - Add vsock support
> #1041 - Idea for easily creating OSv images from list of Ubuntu packages
>
> does manifest_from_host.sh provide most of it?
>
> Fix networking bugs
>
> https://github.com/cloudius-systems/osv/issues/420
> https://github.com/cloudius-systems/osv/issues/454
>
> Faster VFS - https://github.com/cloudius-systems/osv/issues/450
> https://github.com/cloudius-systems/osv/issues/664
> https://github.com/cloudius-systems/osv/issues/589
> Improve XEN support
>
> https://github.com/cloudius-systems/osv/issues/498
> https://github.com/cloudius-systems/osv/issues/345
>
> https://github.com/cloudius-systems/osv/issues/264
> https://github.com/cloudius-systems/osv/issues/235
> Speed-up realloc() by mremap() - 
> https://github.com/cloudius-systems/osv/issues/184
> “Steal-time” to improve scheduler - 
> https://github.com/cloudius-systems/osv/issues/181
> Investigate virtio-net “slow path” - 
> https://github.com/cloudius-systems/osv/issues/177
> Try GOLD linker - https://github.com/cloudius-systems/osv/issues/151
>
> Gold 

[osv-dev] Re: [PATCH V2] threads: allocate application stacks lazily to save memory

2020-06-07 Thread Waldek Kozaczuk
Nadav,

I am not sure if at this point this patch still applies to the source tree at 
this point, but could you please try to see if you can review it as is? 
Otherwise I will revise it with the latest master. 

Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/7f98cc8d-0950-4a6b-abe6-72526de5e293o%40googlegroups.com.


Re: [osv-dev] Next releases - 0.56 and onward - proposal

2020-06-07 Thread Nadav Har'El
On Sat, Jun 6, 2020 at 7:45 PM Waldek Kozaczuk  wrote:

> Hi,
>
> Here is a list of things, not necessarily in any importance order, I am
> proposing we try to accomplish for next releases:
>
>- Prune the list of issues
>   - Close anything that is irrelevant or possibly completed
>  - One subcategory I can think of, are issues about improving
>  java wrapper like this one -
>  https://github.com/cloudius-systems/osv/issues/169. Given we can
>  run unmodified /usr/bin/java without need of a wrapper
>
> Can you please remind me (I haven't been paying enough attention to this
Java stuf...) - is this what we do by default? Are there any downsides to
not using the wrapper?

If we don't use the wrapper by default and there is no reason to go back to
using it, I guess you're right that there is no need to fix any bugs in
that wrapper.


>
>- and given the wrapper (java.so) does not support (or is even
>  compatible) with many features of Java 9 and higher, I suggest we 
> close any
>  issues like 169 which are about improving java.so - I do not think 
> it is
>  worth any time investment (the only real benefit of java.so is 
> ballooning
>  feature which is broken -
>  https://github.com/cloudius-systems/osv/issues/796 and
>  https://github.com/cloudius-systems/osv/issues/1038 [duplicate
>  of 796?] - but if anyone is interested in fixing it I think we might
>  support it without java.so anyway).
>
>
I agree.


>- Better annotate existing issues and especially select ones as '"good
>   first issue" (see
>   
> https://github.blog/2020-01-22-browse-good-first-issues-to-start-contributing-to-open-source/
>   )
>- Better support Linux executables built for Musl-based distributions
>like Alpine (see https://wiki.musl-libc.org/projects-using-musl.html)
>   - These may work even better given OSv libc implementation is to
>   great extent based on musl
>
> If you're talking about dynamically-linked executables, what does it
matter if the shared library in a distribution is glibc or musl?
What is preventing "Alpine" executables from running on OSv? Do we have an
issue about it?


>
>- Tooling
>-
>   - Improve capstan
>   -
>  - Better support QEMU on OSX (new accelerators)
>  - Support pulling artifacts from GitHub nightly releases repo
>  - Support running Firecracker, QEMU microvm
>  - Support Virtio-FS
>   - Create plugins to deploy Java, Node, etc apps
>   -
>  - IntelliJ
>  - Gradle
>  - Maven
>   - Run Docker images (possibly musl-based like Alpine are better)
>  - There is already existing example -
>  
> https://github.com/cloudius-systems/osv-apps/tree/master/openjdk12-jre-from-docker
>   - Review the lazy app thread stack patch aimed to reduce memory
>consumption
>- https://groups.google.com/d/msg/osv-dev/tZnwiScmjZY/GkY0hV9EAwAJ
>- Modularization/"Librarization"
>   - See the discussion -
>   https://groups.google.com/d/msg/osv-dev/BHwN6Stm3n4/arvdOwl3AAAJ
>   - Hide most symbols in the kernel
>   -
>  - Issue #97 - Be more selective on symbols exported from the
>  kernel
>  - See related discussion -
>  https://groups.google.com/d/msg/osv-dev/fs-jmyaXY_E/vWUk-wDDAAAJ
>   - Reduce kernel size:
>   -
>  - See the discussion -
>  https://groups.google.com/d/msg/osv-dev/hCYGRzytaJ4/D23S_ibNAgAJ
>  and
>  https://groups.google.com/d/msg/osv-dev/JsJfnhTX_EY/PKUvYm9-AQAJ
>   - Expose C++ APIs as C to hide C++ completely
>- Keep improving Virtio-FS (#1079)
>-
>   - Integrate with pagecache
>   - Improve DAX
>- Improve ARM support
>   - See this Firecracker patch -
>   https://groups.google.com/d/msg/osv-dev/3TrFxpW9Bhk/AdilzZT5AQAJ
>   - Improve run.py
>- Reach out to Spirent to see if any more networking related bugs
>fixed can be brought over from the https://github.com/SpirentOrion/osv
> fork
>- Add support of Virtio GPU/VGA
>- Upgrade musl
>   - I kind of remember Geraldo Netto started on this
>- Test newer runtimes
>-
>   - Golang >= 1.13
>   - Java > 12
>   - DENO 1.0
>- Support .NET Core executables
>-
>   - See what needs to be finished in this thread -
>   https://groups.google.com/d/msg/osv-dev/k69cHw7qvTg/9NqPq70IAwAJ
>- Refresh look and feel of http://osv.io
>
> Other interesting existing issues:
>
>- Better deal with small amount of memory -
>https://github.com/cloudius-systems/osv/issues/1013
>- #28 - Optimize atomic operations on UP (single-VCPU)
>- #1016 - Add support for smb2/3 (samba) file system T
>   - There is already a patch sent to the list -
>   https://groups.google.com/d/msg/osv-dev/QToC22-QHiA/ejSOMGe7BwAJ
>-
>- #1009 -