Cool! Let's make sure the OpenVPN links on it don't have bufferbloat in them
- VPNs often do, and perhaps need a layer of fq_codel on top of each link (also
underneath, of course, where the routers can be fixed).
More interestingly - is this an openly growing network? Or is it intended to
be an island?
For those stuck in IPv4 land (which includes folks who live inside corporate
Intranets, sadly, maybe this is a good step.
-----Original Message-----
From: "Dave Taht" <[email protected]>
Sent: Wednesday, January 23, 2013 12:11pm
To: [email protected]
Subject: [Cerowrt-devel] An interesting application of tunneling and ipv6 mesh
networking
---------- Forwarded message ----------
From: <[mailto:[email protected]] [email protected]>
Date: Wed, Jan 23, 2013 at 6:30 AM
Subject: [Babel-users] A happy babel user: re6st
To: [mailto:[email protected]]
[email protected]
Hi,
Very often, people complain on mailing lists. Today, I would like to say thank
you.
Last summer, we have implemented a wired mesh network system based on babel
which can provide stable IPv6 to all nodes of a decentralized cloud operation
system. It works great.
Thank you babel.
If you are in a hurry, here is the code:
[http://git.erp5.org/gitweb/re6stnet.git]
http://git.erp5.org/gitweb/re6stnet.git
What you can do with that code: provide reliable IPv6 to the world
If you think re6st is useful, please feel free to add it to the list of babel
links.
Details bellow.
Regards,
JP Smets.
Nexedi CEO
[tel:%2B33%20629%2002%2044%2025] +33 629 02 44 25
---
1- The problem to solve
We implemented a couple of years ago ([http://bit.ly/SWVQlx]
http://bit.ly/SWVQlx) a Cloud system called SlapOS ([http://www.slapos.org]
http://www.slapos.org) which relies on servers located in people's home and now
also in offices, data centers or even your smartphone, tablet or TV. SlapOS is
now used by some large corporations. One of its main applications is to create
a disaster recovery cloud which can resist any force majeur event (ex. war,
terrorism, political instability, software bug) which does affect traditional
clouds from time to time ([http://iwgcr.org] http://iwgcr.org). It is also much
cheaper and environmental friendly.
SlapOS relies on IPv6 in order to interconnect all nodes. Each node is
allocated usually 100 global IPv6 addresses or more.
This is where our problem started: all IPv6 providers we tried were unable to
provide reliable connectivity. We tried providers in France, Germany, Japan,
Norway. For example, in France among 200 IPv6 adresses provided by a Freebox
(Free), 3 becomes unreachable from time to time, during a couple of minutes or
hours. OVH routers sometimes no longer route packers to Free, but only for
IPv6, during a couple of hours. Telia routers somtimes "eat" a few bytes during
the initialization of a session.
Overall, the use of native IPv6 of ISPs lead to a service availability of 99%
or worse. We we searching for a solution.
We also had had the experience that from time to time, IPv4 transit between
ISPs can be cut for a while - a couple of hours -although less often. Our ideal
solution should also solve that.
2- The solution: re6st + babel
Step 1: create a wired mesh
We coded a litlle deaemon called re6st which is able to find 10 IPv4
neighbours randomly and create a tunnel to each neighbour. re6st can be placed
behind a NAT. It is able to capture public IPv4 address of your router through
UPnP. After some time, all nodes which run re6st form a global mesh.
Step 2: start babel
Once tunnels are created, babel is used for routing. Babel then finds the best
route to interconnect all re6st nodes.
3- Results
After a couple of month of using re6st + babel we can say that it works quite
well. SlapOS no longer experiences the connectivity problems of native IPv6. We
can safely host websites with SlapOS over re6st+babel.
4- Next steps
A report will be published.
5- Remaining problems to solve
The problems which remain to be solved are the following:
a- How can we prevent one babel participant to act against other participants
by providing wrong information to other participants ? Imagine for example that
a bad organization joins re6st + babel network and starts capturing all routes
in order to analyze traffic or even block it.
b- How can we create a hierarchical addressing system ? The idea here is to
group participants dynamically and assign them a "big" IPv6 address range. Each
participant connects to another participant through another participant by
first connecting randomly to one participant in a dynamic group and next
connect to other participant in the same group. With this grouping approach,
there is no need to create a hierachical network with a bakbone. It also solves
the problem of scalability.
c- How can we implement more policies (ex. latency) ?
d- How could we implement accounting and billing in a way or another ? (open
question, but quite important for example to solve the problem of FTTH
participants with upload limited to 3GB / day as in Japan)
6- Credits
Most of the coding of re6st was done by Julien Muchembled (Nexedi), Ulysse
Beaugnon (ENS) and Guillaume Bury (ENS).
7- Alternatives
We could have used other routing protocols (ex. OLSR). But we felt that Babel
pluggable policy system was a key design difference which could be used to
later customize it to different needs of Cloud applications (ex. low latency).
We would also feel ashamed to use a protocol which babel's creator proved that
it was flawed.
We could have used tinc. But tinc creates a fully connected mesh. There is
also a difference between what it claims to do and what it actually does. Last,
mixing tunneling and routing is a bad idea as we were suggested by Juliusz C.
We could have used gre instead of OpenVPN for tunnels. But that does work
behind an IPv4 NAT. Yet, nothing prevents use from later using gre.
_______________________________________________
Babel-users mailing list
[mailto:[email protected]] [email protected]
[http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
--
Dave Täht
Fixing bufferbloat with cerowrt:
[http://www.teklibre.com/cerowrt/subscribe.html]
http://www.teklibre.com/cerowrt/subscribe.html
_______________________________________________
Cerowrt-devel mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/cerowrt-devel
