Re: 'Live' Migrate messes up NTP on FreeBSD domU - any suggestions?

[Wei Liu]

On Mon, Jul 25, 2016 at 04:43:43PM +0200, Roger Pau Monné wrote:
> Adding Wei to the Cc list since he added the multiqueue functionality.
> 
> On Mon, Jul 25, 2016 at 02:59:02PM +0100, Karl Pielorz wrote:
> > 
> > --On 22 July 2016 13:55 +0200 Roger Pau Monné  wrote:
> > 
> > > In my environment I've migrated a FreeBSD VM with 2 cpus for > 100
> > > consecutive times without seeing any issues (or freezes), although this
> > > was  with OSS Xen and without xe-guest-utilities. Karl, have you tested
> > > HEAD  recently?
> > 
> > Ok, I have tested this with r303286 - it seems to work OK. The hosts gain no
> > time that I can see while migrating, and NTP stays happy.
> > 
> > I did get a panic after about 40 migrations - but that seems to be some
> > network issue or something...
> > 
> >   ('panic called with 0 available queues / dbt_trace_self_wrapper / vpanic /
> > kassert_panic / xn_txq_mq_start / ether_output / udp_send / sosend_dgram /
> > kern_sendit / sendit / sys_sendto / amd64_syscall / Xfast_syscall)
> 
> I haven't been able to reproduce this, but I think it's possible that if you 
> migrate an active netfront xn_txq_mq_start might be called during the 
> migration, just in the middle of the setup_device reconfiguation (while 
> info->num_queues is 0).
> 
> Wei, I think netif_disconnect_backend should set IFF_DRV_OACTIVE in order to 
> notify the net subsystem that the queues are full, so no further calls to 
> xn_txq_mq_start happen until the resume has finished, do you agree?
> 

Perhaps clear IFF_DRV_RUNNING and only set it when the device is ready?
Looking at the manpage is seems more appropriate to me semantically.

Wei.

> Roger.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: [Xen-devel] HEADS UP: Imported Xen 4.7: no blkback

[Wei Liu]

On Thu, Jun 09, 2016 at 10:03:43AM +0200, Roger Pau Monné wrote:
> On Thu, Jun 09, 2016 at 12:16:59AM +, Marcin Cieslak wrote:
> > On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> > 
> > > One of the more relevant changes in 4.7 regarding FreeBSD is the support 
> > > for 
> > > block hotplug scripts. This means that we now have the option to use 
> > > backends different than simple block or regular files, provided that 
> > > someone 
> > > writes the proper hotplug scripts to attach them (I've heard there are 
> > > some 
> > > iSCSI hotplug scripts around). This however requires changes in blkback, 
> > > so 
> > > if you plan to use the Xen 4.7 port, please make sure that you are 
> > > running a 
> > > kernel that contains revision r301269 (or any later version). The same 
> > > also 
> > 
> > I am running it with r301685 and the HVM guests have some trouble with
> > block devices.
> > 
> > SeaBIOS does not find /dev/zvol/zroot/freebsd1,raw,xvda,w to boot FreeBSD
> > from, after chaging to "hda" I get up to the kernel mountroot prompt
> > (Xen block devices seem to be detected in dmesg).
> 
> Yes, this is intentional, see:
> 
> https://marc.info/?l=xen-devel=144482080812353
> 

Note that I reverted that change yesterday. 4.7 will behave as before.

Wei.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: xn ethernet issues as DOMU under NetBSD DOM0

[Wei Liu]

I've only gotten two emails on this, so forgive me if this has been
discussed.

On Sat, May 07, 2016 at 04:11:52PM +, Stephen Jones wrote:
> From: kmacy...@gmail.com [mailto:kmacy...@gmail.com] On Behalf Of K. Macy
> >That would explain it. If the Linux backend supports TSO, then netfront will 
> >of course advertise TSO. And >presumably there's no way to query the 
> >netback, since Xen is linux-centric. Assuming you haven't already I would 
> >>disable TSO.
> 
> > Or have you already tried that?
> 
> This morning I've tried building a kernel without the TCP_OFFLOAD option.  
> When booting the new system, I see this in dmesg:
> 
> xn0:   at device/vif/0 on xenbusb_front0
> xn0:  Ethernet address:  00:16:3e:00:00:30
> xn0:  backend features:  feature-sg feature-gso-tcp4
> xn_txeof:  WARNING:  response is -1!
> 

This means the backend returns -1 for the tx request, but -1 is just
a general error code so it doesn't reveal much.

It would be interesting to see if there is anything shown on the backend
side -- if netbsd's netback logs something.

> Ifconfig -v xn0 looks like:
> 
> xn0:  flags=8843 metric 0 mtu 1500
>   options=403
> 
> Pings work, ftp works (I was able to get the src.txz file off of 
> ftp.freebsd.org to build the kernel)
> ssh does not work (handshaking fails) but I can login via telnet.  However, 
> just about anything
> that requires lots of output gets a bit clobbered.
> 
> Are there any other kernel/sysctl parameters I should be disabling?
> 
> It is almost working, but I need help to know if this is a netfront or a 
> netback issue.  If it is a NetBSD issue
> I'll move the discussion over to port-...@netbsd.org

I think what you can do is to try the same kernel with a Linux backend.
That would be a good way of telling which side is at fault.

Wei.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: Xen/dom0/FreeBSD + NAS4Free WebGUI.

[Wei Liu]

On Tue, Jan 12, 2016 at 02:23:39PM +0100, Roger Pau Monné wrote:
[...]
> I'm adding Wei to the Cc, he has been working on netfront improvements,
> so maybe he also wants to take a stab at netback ;).

It's on my radar but I don't have time for it in the near future. If
anyone else who is watching FreeBSD Xen implementation have the desire
to improve it I'm happy to help.

Wei.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: Checksum forwarding issue on XEN

[Wei Liu]

On Thu, Nov 05, 2015 at 05:30:06PM +0100, Roger Pau Monné wrote:
> El 05/11/15 a les 17.00, Larry Baird ha escrit:
> > Roger,
> > 
> >> Adding the persons that contributed that code in case they can shed some
> >> light.
> >>
> >> El 03/11/15 a les 21.12, Larry Baird ha escrit:
> >>> Has anybody made any progress on "Bug 188261 - [xen] FreeBSD DomU PVHVM
> >>> guests cannot 'route' traffic for other Xen PV guests on same Dom0 Host."
> >>> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=188261)?
> >>>
> >>> The code for checksum calculation in the function xnb_add_mbuf_cksum() 
> >>> looks
> >>> suspect.
> >>>
> >>> switch (iph->ip_p) {
> >>> case IPPROTO_TCP:
> >>> if (mbufc->m_pkthdr.csum_flags & CSUM_IP_VALID) {
> >>> size_t tcplen = ntohs(iph->ip_len) - 
> >>> sizeof(struct ip);
> >>> struct tcphdr *th = (struct tcphdr*)(iph + 1);
> >>> th->th_sum = in_pseudo(iph->ip_src.s_addr,
> >>> iph->ip_dst.s_addr, htons(IPPROTO_TCP + 
> >>> tcplen));
> >>> th->th_sum = in_cksum_skip(mbufc,
> >>> sizeof(struct ether_header) + 
> >>> ntohs(iph->ip_len),
> >>> sizeof(struct ether_header) + (iph->ip_hl << 
> >>> 2));
> >>> }
> >>> break;
> >>> case IPPROTO_UDP:
> >>> if (mbufc->m_pkthdr.csum_flags & CSUM_IP_VALID) {
> >>> size_t udplen = ntohs(iph->ip_len) - 
> >>> sizeof(struct ip);
> >>> struct udphdr *uh = (struct udphdr*)(iph + 1);
> >>> uh->uh_sum = in_pseudo(iph->ip_src.s_addr,
> >>> iph->ip_dst.s_addr, htons(IPPROTO_UDP + 
> >>> udplen));
> >>> uh->uh_sum = in_cksum_skip(mbufc,
> >>> sizeof(struct ether_header) + 
> >>> ntohs(iph->ip_len),
> >>> sizeof(struct ether_header) + (iph->ip_hl << 
> >>> 2));
> >>> }
> >>> break;
> >>> default:
> >>> break;
> >>> }
> >>>
> >>>
> >>> Both in_pseudo() and in_cksum_skip() set the same checksum. Does this
> >>> make since to anybody?
> >>
> >> The bug you are referring to affects FreeBSD when running as a guest
> >> using xen-netfront, but the code snipped above and the function
> >> referenced (xnb_add_mbuf_cksum) is only used on FreeBSD when running as
> >> a host (AKA Dom0) by xen-netback.
> >>
> >> TBH, I don't know that much about FreeBSD network subsystem to have an
> >> opinion, but it certainly looks weird. Patches are welcome :).
> > 
> > Xyper-V has a similar forward issue. I found they were misusing csum_flags
> > and were always attempting to do checksum offloading if CSUM_IP_VALID was
> > set. I have given them a patch that fixes the issue. I was hoping that
> > Xen's issue was similar.  I found the issue above by looking at all uses
> > of csum_flags in sys/dev/xen. It is hard to tell what the correct fix
> > is, without fulling understand the protocal used when communicating between
> > backend and frontend of Xen. 
> 
> 
> 
> > I am sure issue with XEN guest forwarding has to with checksum offloading.
> > If I am not misinterpreting your comments, I can ignore code in netback and
> > concentrate on code in netfront when trying to understand what is going 
> > wrong.
> 
> Yes, this issue is related to netfront (sys/dev/xen/netfront/netfront.c)
> only, netback code is not involved. The code related to the checksum
> stuff is on line ~1409 for the TX side, and around line 872 for the RX
> side AFAICT.
> 
> You can find more information about the protocol itself in
> sys/xen/interface/io/netif.h.
> 
> Adding Wei Liu who is also doing some work to improve netfront, and
> knows more about the protocol than myself.
> 

Sorry for the late reply.

I think the place to look at, as Roger suggested, is netfront checksum
setup code.

The relevant flags in Xen network protocol are:

XEN_NETTXF_csum_blank -- protocol checksum field is blank
XEN_NETTXF_data_validated -- data has been validated

I'm not entirely sure FreeBSD netfront is mapping its own CSUM_* flags
correctly to Xen's.

I think at some point I can look into it, but not in the near future.

Wei.


> Roger.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"