Re: [osv-dev] IPV6 branch

2019-12-12 Thread Dor Laor 
Waldek, it's great that OSv is in such good hands!
With the growth of serverless and the shrinkage in qemu+osv boot time, it
opens lots of more use cases for our users.
Dor

On Thu, Dec 12, 2019 at 3:03 PM Waldek Kozaczuk 
wrote:

> Hi,
>
> Yesterday I merged the IPV6 patches submitted by Charles Myers from
> Spirent over a year ago -
> https://groups.google.com/d/msg/osv-dev/WJ52lvQtC9Q/gCyNWsO0CwAJ - into a
> new branch *ipv6* (https://github.com/cloudius-systems/osv/tree/ipv6)
> based on the current master branch. These patches add IPV6 support as well
> as fix some bugs in OSv networking code which is based on FreeBSD 9
> codebase.
>
> The sheer breadth of the patches is enormous and gives a credit to the
> excellent work by Charles (great job!!!). Given that Nadav reviewed them
> and was generally happy with them, I thought it would be a shame if we let
> it get forgotten like this. Now, given it touches very sensitive and
> complicated networking code, I thought it would be wise to put it on a
> separate branch, test it more by the community and then merge it into
> master. Given the networking code does not change that often, we could
> somehow auto-merge master into it (does github offer such ability?).
>
> Most of the patches were merged without conflicts using 'git am -m', in
> some few cases I had to resolve some conflicts:
>
>- virtio-net, vmxnet code -
>
> https://github.com/cloudius-systems/osv/commit/ee7b0ef05016819fbb9eb0e9dc29ff343672af81
>- loader.cc where we replaced boost::program_options -
>
> https://github.com/cloudius-systems/osv/commit/723893512fefe58a9a86d1c7aab52fb99140b83a
>- unit tests Makefile -
>
> https://github.com/cloudius-systems/osv/commit/87fb81b91cd2d7fcbe11ca8418df444afa6ea430
>- minor conflicts in the main Makefile -
>
> https://github.com/cloudius-systems/osv/commit/47381fb07bad3274f411baeb0c39b2a96341aaec
>
> I did not apply one patch -
> https://groups.google.com/d/msg/osv-dev/WJ52lvQtC9Q/HAQW4y60BQAJ - with
> cloud init support - which seems to somewhat independent. Rick Payne
> mentioned that he might send a new version that addresses also the program
> options removal implications.
>
> I also had to fix number of gcc 8/9 compiler warnings by either adjusting
> the code to use macros I added to deal with packed struct member address
> warnings or disabling certain warnings (-fno-strict-aliasing,
> no-address-of-packed-member):
>
>-
>
> https://github.com/cloudius-systems/osv/commit/2ead349dc560a02b412154664420cf1ac0c00f01
>-
>
> https://github.com/cloudius-systems/osv/commit/cef7d8e2403cf8f5f96b38fb7fe5a8e78683b657
>-
>
> https://github.com/cloudius-systems/osv/commit/87522701c4d0844a568c762e675aca9ce27e
>
> If anyone could review those it would be great.
>
> I was able to compile and run the unit tests  (*all pass!*) with IPV6
> enabled (did not test when disabled) on Ubuntu 19.10 with gcc 9.2. I also
> stress-tested a little bit the networking layer with httpserver apps like
> golang-httpserver and rust-httpserver as well as with iperf3. It all seems
> to work. I did not really test any IPV6 scenarios yet (except for what the
> new unit tests that Charles added do). *I am not very familiar with the
> setup with QEMU, ipv6 itself so that is where I am asking for help to test!*
>
> I have also noticed that based on requests per second measured with apache
> bench and iperf3 the ipv6 branch seems to be 3-5% slower than master (ipv4
> traffic but with ipv6 code enabled). But this is just the observation, not
> a hard measurement. But again code seems to work reliably and correctly for
> what my testing is worth.
>
> Finally looking at the Spirent fork (https://github.com/SpirentOrion/osv)
> I think they have made some more changes to the networking code after those
> patches were submitted. It would be nice to bring them over.
>
> Waldek
>
> PS. Some code statistics:
>
>- 38K new lines of code (wc -l bsd/sys/netinet6/*)
>- the loader-stripped.elf binary (kernel) size increases by ~270K with
>IPV6 enabled; I think it is worth price to pay given the functionality
>added (plus we have a witch to turn it on/off); recently by removing
>boost-programs options and replacing with hand-written equivalent let us
>save around 300-500K; so nice exchange!
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/CAL9cFfNNGgXKBkkQKfx%3D7LDz%3DyqG5iR6Um2n5sHWvEhZAb7VrA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development"

[osv-dev] IPV6 branch

2019-12-12 Thread Waldek Kozaczuk 
Hi,

Yesterday I merged the IPV6 patches submitted by Charles Myers from Spirent
over a year ago -
https://groups.google.com/d/msg/osv-dev/WJ52lvQtC9Q/gCyNWsO0CwAJ - into a
new branch *ipv6* (https://github.com/cloudius-systems/osv/tree/ipv6) based
on the current master branch. These patches add IPV6 support as well as fix
some bugs in OSv networking code which is based on FreeBSD 9 codebase.

The sheer breadth of the patches is enormous and gives a credit to the
excellent work by Charles (great job!!!). Given that Nadav reviewed them
and was generally happy with them, I thought it would be a shame if we let
it get forgotten like this. Now, given it touches very sensitive and
complicated networking code, I thought it would be wise to put it on a
separate branch, test it more by the community and then merge it into
master. Given the networking code does not change that often, we could
somehow auto-merge master into it (does github offer such ability?).

Most of the patches were merged without conflicts using 'git am -m', in
some few cases I had to resolve some conflicts:

   - virtio-net, vmxnet code -
   
https://github.com/cloudius-systems/osv/commit/ee7b0ef05016819fbb9eb0e9dc29ff343672af81
   - loader.cc where we replaced boost::program_options -
   
https://github.com/cloudius-systems/osv/commit/723893512fefe58a9a86d1c7aab52fb99140b83a
   - unit tests Makefile -
   
https://github.com/cloudius-systems/osv/commit/87fb81b91cd2d7fcbe11ca8418df444afa6ea430
   - minor conflicts in the main Makefile -
   
https://github.com/cloudius-systems/osv/commit/47381fb07bad3274f411baeb0c39b2a96341aaec

I did not apply one patch -
https://groups.google.com/d/msg/osv-dev/WJ52lvQtC9Q/HAQW4y60BQAJ - with
cloud init support - which seems to somewhat independent. Rick Payne
mentioned that he might send a new version that addresses also the program
options removal implications.

I also had to fix number of gcc 8/9 compiler warnings by either adjusting
the code to use macros I added to deal with packed struct member address
warnings or disabling certain warnings (-fno-strict-aliasing,
no-address-of-packed-member):

   -
   
https://github.com/cloudius-systems/osv/commit/2ead349dc560a02b412154664420cf1ac0c00f01
   -
   
https://github.com/cloudius-systems/osv/commit/cef7d8e2403cf8f5f96b38fb7fe5a8e78683b657
   -
   
https://github.com/cloudius-systems/osv/commit/87522701c4d0844a568c762e675aca9ce27e

If anyone could review those it would be great.

I was able to compile and run the unit tests  (*all pass!*) with IPV6
enabled (did not test when disabled) on Ubuntu 19.10 with gcc 9.2. I also
stress-tested a little bit the networking layer with httpserver apps like
golang-httpserver and rust-httpserver as well as with iperf3. It all seems
to work. I did not really test any IPV6 scenarios yet (except for what the
new unit tests that Charles added do). *I am not very familiar with the
setup with QEMU, ipv6 itself so that is where I am asking for help to test!*

I have also noticed that based on requests per second measured with apache
bench and iperf3 the ipv6 branch seems to be 3-5% slower than master (ipv4
traffic but with ipv6 code enabled). But this is just the observation, not
a hard measurement. But again code seems to work reliably and correctly for
what my testing is worth.

Finally looking at the Spirent fork (https://github.com/SpirentOrion/osv) I
think they have made some more changes to the networking code after those
patches were submitted. It would be nice to bring them over.

Waldek

PS. Some code statistics:

   - 38K new lines of code (wc -l bsd/sys/netinet6/*)
   - the loader-stripped.elf binary (kernel) size increases by ~270K with
   IPV6 enabled; I think it is worth price to pay given the functionality
   added (plus we have a witch to turn it on/off); recently by removing
   boost-programs options and replacing with hand-written equivalent let us
   save around 300-500K; so nice exchange!

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfNNGgXKBkkQKfx%3D7LDz%3DyqG5iR6Um2n5sHWvEhZAb7VrA%40mail.gmail.com.