[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #20 from Craig Rodrigues  ---
Sorry for the late feedback.  I can confirm that you fixed the problem.
This was a very weird and convoluted corner case.  Thank you so
much for digging into this and fixing it!!

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

Michael Tuexen tue...@freebsd.org changed:

   What|Removed |Added

 Status|In Progress |Closed
 Resolution|--- |FIXED

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #19 from commit-h...@freebsd.org ---
A commit references this bug:

Author: tuexen
Date: Sat Jun 20 08:25:31 UTC 2015
New revision: 284633
URL: https://svnweb.freebsd.org/changeset/base/284633

Log:
  MFC r284515:
  Add FIB support for SCTP.
  This fixes https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

  PR:200379

Changes:
_U  stable/10/
  stable/10/sys/netinet/sctp_asconf.c
  stable/10/sys/netinet/sctp_input.c
  stable/10/sys/netinet/sctp_input.h
  stable/10/sys/netinet/sctp_os_bsd.h
  stable/10/sys/netinet/sctp_output.c
  stable/10/sys/netinet/sctp_output.h
  stable/10/sys/netinet/sctp_pcb.c
  stable/10/sys/netinet/sctp_pcb.h
  stable/10/sys/netinet/sctp_usrreq.c
  stable/10/sys/netinet/sctputil.c
  stable/10/sys/netinet/sctputil.h
  stable/10/sys/netinet6/sctp6_usrreq.c

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #18 from Michael Tuexen tue...@freebsd.org ---
(In reply to Craig Rodrigues from comment #17)
Ahh. Thanks for the hint. Will do.

Best regards
Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #16 from Michael Tuexen tue...@freebsd.org ---
(In reply to Alan Somers from comment #15)
Yes, you can failover from one ISP to another. Currently this is done by having
corresponding entries in a single routing table for the multiple peer
addresses.
I have checked in support for FIB support in
https://svnweb.freebsd.org/changeset/base/284515
This is a single fib per socket. This way you can have multiple applications on
a single host using SCTP and they can have individual setups. Better than the
current
situation.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #14 from Michael Tuexen tue...@freebsd.org ---
(In reply to Alan Somers from comment #13)
I think interfaces can assign fibs to packets, it is a field in the mbuf packet
header. It makes sense to use this information in case you have no socket to
get the fib from (for example when receiving a TCP SYN and you have no
listening socket).

An SCTP end-point can have multiple IP addresses. When using multihoming you
use multiple local and remote IP-addresses to provide network fault tolerant.
So you use multiple local interfaces and route traffic on all of them to be
able to fail over in case of network problems. Of course you can setup this in
a single routing table and have a socket in a single fib. I'm tending to
implement it this way. This also means that for response packets (like acks for
data) use the socket's fib, not the one from the incoming packet. At least this
is conceptually simpler. Codewise it doesn't make much of a difference.

Thanks for your feedback.

Best regards
Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #15 from Alan Somers asom...@freebsd.org ---
(In reply to Michael Tuexen from comment #14)

You're right about the interface FIB.  It will take incoming packets with a
certain FIB.  But it's not completely general; it's possible to have outbound
traffic use multiple FIBs on a single interface.

The part about multihoming is more interesting.  Can you use SCTP to failover
from one ISP to another?  Different ISPs require different gateways, and hence
different routing tables.  In that case, a single fib per SCTP socket wouldn't
be sufficient.  We would need to set the FIB separately for each local IP
address of the SCTP socket.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #13 from Alan Somers asom...@freebsd.org ---
FIBs are used to have different routing policies for different kinds of
traffic.  In general, you can't correctly learn which fib you ought to use
based on any feature of a received packet, because different applications can
use different FIBs at the same time.  One application can even use more than
one FIB.

I don't know much about SCTP, but I think that there should be a FIB per
socket.  That's what the socket API currently allows, and it makes intuitive
sense.  Would you ever want to have multiple paths of the same socket get
routed out different interfaces or to different gateways?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #12 from Michael Tuexen tue...@freebsd.org ---
Hi Craig,

I'm in the process of understanding how fibs work and thinking about how they
should work for SCTP. So one possibility is that a socket uses a fib. So all
paths of an SCTP association will use the same fib. Another possibility is to
allow a fib per path. Especially, SCTP would learn a fib from incoming
packets. Do you have an opinion which one is more appropriate? Any reasons to
share?

Best regards
Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #11 from Michael Tuexen tue...@freebsd.org ---
Yes, I do. Using net.add_addr_allfibs=0, I get routing tables like you have.
However, I had to call setfib() before socket() in your examples. With that I
can reproduce the problem.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #5 from Michael Tuexen tue...@freebsd.org ---
Hi Craig,

when setting up two VMs as suggested, they can just reach each other.
Even ping 172.8.1.4 works, I don't need setfib 2 ping 172.8.1.4.

What config do I need to test the fib stuff. I don't think storing it
in the inp is the way, since this values wouldn't be updated if a
setsockopt() operation is performed. So I think just using the fibnum
from the socket in the SCTP_RTALLOC macro is the way to go. But I want
to test it before committing it.

Best regards
Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #7 from Michael Tuexen tue...@freebsd.org ---
Here is what I do and what happens to the routing table. As you see, a route
gets added to fib 0. Is this expected? Intended?

 ifconfig em0
em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
options=9bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM
ether 00:0c:29:8a:89:10
inet 192.168.115.171 netmask 0xff00 broadcast 192.168.115.255 
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
media: Ethernet autoselect (1000baseT full-duplex)
status: active
 netstat -nrfinet
Routing tables

Internet:
DestinationGatewayFlags Netif Expire
default192.168.115.2  UGS em0
127.0.0.1  link#2 UH  lo0
192.168.115.0/24   link#1 U   em0
192.168.115.171link#1 UHS lo0
 sudo ifconfig em0.3275 create name craig0 fib 2
Password:
 netstat -nrfinet
Routing tables

Internet:
DestinationGatewayFlags Netif Expire
default192.168.115.2  UGS em0
127.0.0.1  link#2 UH  lo0
192.168.115.0/24   link#1 U   em0
192.168.115.171link#1 UHS lo0
 sudo ifconfig craig0 inet6 ifdisabled fib 2
 netstat -nrfinet
Routing tables

Internet:
DestinationGatewayFlags Netif Expire
default192.168.115.2  UGS em0
127.0.0.1  link#2 UH  lo0
192.168.115.0/24   link#1 U   em0
192.168.115.171link#1 UHS lo0
 sudo ifconfig craig0 inet 172.8.1.3/16 up fib 2
 sudo ifconfig craig0 
craig0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
options=3RXCSUM,TXCSUM
ether 00:0c:29:8a:89:10
inet 172.8.1.3 netmask 0x broadcast 172.8.255.255 
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
media: Ethernet autoselect (1000baseT full-duplex)
status: active
fib: 2
vlan: 3275 parent interface: em0
groups: vlan 
 netstat -nrfinet
Routing tables

Internet:
DestinationGatewayFlags Netif Expire
default192.168.115.2  UGS em0
127.0.0.1  link#2 UH  lo0
172.8.0.0/16   link#3 Ucraig0
192.168.115.0/24   link#1 U   em0
192.168.115.171link#1 UHS lo0
 setfib 2 netstat -nrfinet
Routing tables (fib: 2)

Internet:
DestinationGatewayFlags Netif Expire
127.0.0.1  link#2 UH  lo0
172.8.0.0/16   link#3 Ucraig0
172.8.1.3  link#3 UHS lo0
192.168.115.0/24   link#1 U   em0
 sudo route add 127.0.0.0/8 -interface lo0 -fib 2
add net 127.0.0.0: gateway lo0 fib 2
 setfib 2 netstat -nrfinet
Routing tables (fib: 2)

Internet:
DestinationGatewayFlags Netif Expire
127.0.0.0/8lo0US  lo0
127.0.0.1  link#2 UH  lo0
172.8.0.0/16   link#3 Ucraig0
172.8.1.3  link#3 UHS lo0
192.168.115.0/24   link#1 U   em0
 netstat -nrfinet
Routing tables

Internet:
DestinationGatewayFlags Netif Expire
default192.168.115.2  UGS em0
127.0.0.1  link#2 UH  lo0
172.8.0.0/16   link#3 Ucraig0
192.168.115.0/24   link#1 U   em0
192.168.115.171link#1 UHS lo0

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #6 from Craig Rodrigues rodr...@freebsd.org ---
You need to set up a default IP address and routing table
on em0 that is *not* a 172 address.

That way, if you do:

netstat -r

you will see the default routing table,
and if you do

setfib 2 netstat -r

you will see the routing table for the 172 addresses

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #10 from Craig Rodrigues rodr...@freebsd.org ---
(In reply to Michael Tuexen from comment #8)
Are you good to go with having an environment to repro the problem?

My routing table looks like this

FIB 0
=
netstat -nr

default10.47.1.1  UGS  vtnet2
10.47.0.0/16   link#3 Uvtnet2
10.47.250.26   link#3 UHS lo0
127.0.0.1  link#5 UH  lo0



FIB 2
=
setfib 2 netstat -nr

Routing tables (fib: 2)

Internet:
DestinationGatewayFlags  Netif Expire
127.0.0.0/8lo0US  lo0
172.8.0.0/16 link#6 U  craig0
172.8.1.3   link#6 UHS lo0

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

Michael Tuexen tue...@freebsd.org changed:

   What|Removed |Added

 Status|New |In Progress

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #8 from Michael Tuexen tue...@freebsd.org ---
OK, I need 
sysctl -w net.add_addr_allfibs=0
to reproduce your problem.

Best regards
Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

Alan Somers asom...@freebsd.org changed:

   What|Removed |Added

 CC||asom...@freebsd.org

--- Comment #9 from Alan Somers asom...@freebsd.org ---
BTW, I wrote some FIB-related tests in tests/sys/netinet.  They use tap(4)
interfaces so they don't need two machines.  If you look at the udp_dontroute
test, you can see an example that passes traffic from one to the other.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #4 from Michael Tuexen tue...@freebsd.org ---
Hi Craig,

thank you very much for reporting the issue and providing steps to reproduce
the problem. I'll look into it.

Best regards
Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

Craig Rodrigues rodr...@freebsd.org changed:

   What|Removed |Added

   Assignee|freebsd-b...@freebsd.org|tue...@freebsd.org
 CC||freebsd-net@FreeBSD.org,
   ||r...@freebsd.org

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #3 from Craig Rodrigues rodr...@freebsd.org ---
In sys/netinet/sctp_os_bsd.h , 
the SCTP_RTALLOC macro calls the rtalloc_ign() function which ignores fibs.

It should probably be changed to call rtalloc_ign_fib()

In addition, it may be necessary to store the fib_num in the
inp and inherit it when accepting/peelingoff.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #1 from Craig Rodrigues rodr...@freebsd.org ---
Created attachment 157024
  -- https://bugs.freebsd.org/bugzilla/attachment.cgi?id=157024action=edit
server.c

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org

[Bug 200379] SCTP stack is not FIB aware

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200379

--- Comment #2 from Craig Rodrigues rodr...@freebsd.org ---
Created attachment 157025
  -- https://bugs.freebsd.org/bugzilla/attachment.cgi?id=157025action=edit
client.c

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org