[Xen-ia64-devel] RE: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
From: Keir Fraser [mailto:[EMAIL PROTECTED] Sent: 2006年3月2日 16:26 On 2 Mar 2006, at 05:43, Tian, Kevin wrote: However the interesting thing is, following Cset is only for changing way to map dom0's store page, instead of domU. DomU's store page is still mapped by foreign page map. If above hint is real cause, xend start can fail earlier due to incorrect mapping when introducing dom0 into xenstore. However you all observe the bug bothering only when domU is booting... The dom0 connection is only used in anger when first creating a domU, to set up device backend state. Until that time it could be screwed and you probably would not notice. -- Keir Thanks. Make sense. -Kevin ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] RE: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
Hi, Kevin
Thank you for your advice.
I tried to compile define ARCH_HAS_DEV_MEM.
but nothing changed.
Best Regards,
Akio Takebe
Hi, Akio,
Currently linux-2.6-xen-sparse/driver/xen/char/ is not included
in compilation for xen/ia64, so you're still using linux-style kmem
path.
Try to compile that directory into your xenlinux image, and
define ARCH_HAS_DEV_MEM, and then see anything changed
for you.
However the interesting thing is, following Cset is only for
changing way to map dom0's store page, instead of domU. DomU's
store page is still mapped by foreign page map. If above hint is real
cause, xend start can fail earlier due to incorrect mapping when
introducing dom0 into xenstore. However you all observe the bug
bothering only when domU is booting...
Thanks,
Kevin
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Akio Takebe
Sent: 2006年3月2日 8:44
To: Xu, Anthony; Magenheimer, Dan (HP Labs Fort Collins); xen-devel
Cc: yo.fujita; Akio Takebe; xen-ia64-devel@lists.xensource.com
Subject: RE: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
Hi, Anthony
Thank you for your advice.
I checked retun value of mmap(), and it is not NULL.
I'll check vcpu_translate().
Best Regards,
Akio Takebe
It is likely some subtle difference (or bug), perhaps in mmap?
As I know, in Redhat, mmap can return NULL address, but seems xen
can't handle this situation, see below code segment:
In function vcpu_translate() of vcpu.c file
else if (!region warn_region0_address) {
REGS *regs = vcpu_regs(vcpu);
unsigned long viip = PSCB(vcpu,iip);
unsigned long vipsr = PSCB(vcpu,ipsr);
unsigned long iip = regs-cr_iip;
unsigned long ipsr = regs-cr_ipsr;
printk(vcpu_translate: bad address %p, viip=%p, vipsr=%p, iip=%p,
ipsr=%p continuing\n, address, viip, vipsr, iip, ipsr);
}
warn_region0_address is turned off by default,
so maybe we can turn on warn_region0_address to see whether this is the
root cause.
Thanks,
-Anthony
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
Magenheimer, Dan
(HP Labs Fort Collins)
Sent: 2006ト・ヤツ1ネユ 5:45
To: xen-devel
Cc: yo.fujita; Akio Takebe; xen-ia64-devel@lists.xensource.com
Subject: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
Hi all --
We are seeing a strange problem where a recent cset causes
Red Hat to fail domU boot on ia64 complaining of a hotplug
problem but doesn't cause any problem for Suse or Debian.
It is likely some subtle difference (or bug), perhaps in mmap?
Suggestions/advice from anyone more familiar with distro
differences (on ia64) would be appreciated.
Changeset is xen-unstable 8783 (Use /dev/kmem to map dom0
xenstore page instead of abusing the foreign mapping interface.,
Feb 8, committed by Christian). Backing out this changeset
or using the small patch below causes the problem to go away,
so we have a workaround, but a root cause would be nice
to know and fix.
Full thread of discussion can be found here:
http://lists.xensource.com/archives/html/xen-ia64-devel/2006-02/msg00104
.html
Thanks,
Dan
-Original Message-
From: Akio Takebe [mailto:[EMAIL PROTECTED]
Sent: Tuesday, February 28, 2006 2:47 AM
To: Magenheimer, Dan (HP Labs Fort Collins); yo.fujita;
xen-ia64-devel@lists.xensource.com
Cc: Akio Takebe
Subject: RE: [Xen-ia64-devel] Weekly benchmark results [2/3rd week]
Hi, Dan
I'm still debuging, but it is very difficult...
Much advice is welcome. :-)
Now I can boot domU by using the following patch.
diff -r 6c43118bdba8 tools/xenstore/xenstored_domain.c
--- a/tools/xenstore/xenstored_domain.c Fri Feb 24 15:41:08 2006 -0700
+++ b/tools/xenstore/xenstored_domain.c Tue Feb 28 18:20:16 2006
+0900
@@ -467,6 +467,7 @@ static int dom0_init(void)
int rc, fd;
evtchn_port_t port;
unsigned long kva;
+ unsigned long mfn;
char str[20];
struct domain *dom0;
@@ -500,9 +501,16 @@ static int dom0_init(void)
if (fd == -1)
return -1;
- dom0-interface = mmap(NULL, getpagesize(),
PROT_READ|PROT_WRITE,
- MAP_SHARED, fd, kva);
- if (dom0-interface == MAP_FAILED)
+ mfn=((0x0fff kva) 14);
+/*
+dom0-interface = mmap(NULL, getpagesize(),
PROT_READ|PROT_WRITE,
+ MAP_SHARED, fd, kva);
+*/
+ dom0-interface = xc_map_foreign_range(
+ *xc_handle, 0,
+ getpagesize(), PROT_READ|PROT_WRITE, mfn);
+ if (!dom0-interface)
+// if (dom0-interface == MAP_FAILED)
goto outfd;
close(fd);
Best Regards,
Akio Takebe
Hi Akio --
Any more progress on this issue? If you are stuck,
maybe we should post the problem to xen-devel to
see if we can get help from a Red Hat person (since
the problem doesn't occur on Suse or Debian).
Re: [Xen-ia64-devel] RE: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
On Thu, 2006-03-02 at 18:41 +, Ewan Mellor wrote: These are requests from the xenbus driver to create watches to monitor for new devices being created. That presumably means that either your xenbus driver is fubar'd, or the mmap'd page is bust, as discussed earlier. The requests that you are seeing are from xenconsoled and xend, each of which is using the unix domain socket to talk to the store, not the shared page. If I dump the xsd_kva page using mmap from another app, there are a few entries made: : 01 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 0010: 64 65 76 69 63 65 00 00 00 00 00 00 00 00 00 00 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... 0400: 10 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 0410: 45 4e 4f 45 4e 54 00 00 00 00 00 00 00 00 00 00 0420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... 0800: 17 00 00 00 17 00 00 00 00 00 00 00 17 00 00 00 As you indicate, lots of output in the xenstored-trace file, but hardly anything here. We are getting some transactions in there though, so maybe we're dealing with a memory ordering issue. Thanks, Alex -- Alex Williamson HP Linux Open Source Lab ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Re: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
Le Mercredi 01 Mars 2006 02:30, Xu, Anthony a écrit : It is likely some subtle difference (or bug), perhaps in mmap? Hi all, yesterday I got the same bug as Akio. Today it is working again. I have just reinstalled tools+xen+kernel. Akio, could you try to reinstall xen+tools+kernel using a clean repository ? (make clean may be not enough). It seems *really* strange that only a few of us got the bug because it seems to be related only to Xen+Xenstore+Linux. Therefore it should not depend on the distribution. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Re: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
Hi, Tristan Thank you. I'll try to reinstall soon. If reinstall is a root cause, it is mean some tools are not overwrite when we run make install-tools. Best Regards, Akio Takebe Le Mercredi 01 Mars 2006 02:30, Xu, Anthony a ィヲcrit : It is likely some subtle difference (or bug), perhaps in mmap? Hi all, yesterday I got the same bug as Akio. Today it is working again. I have just reinstalled tools+xen+kernel. Akio, could you try to reinstall xen+tools+kernel using a clean repository ? (make clean may be not enough). It seems *really* strange that only a few of us got the bug because it seems to be related only to Xen+Xenstore+Linux. Therefore it should not depend on the distribution. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] RE: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
From: Akio Takebe Sent: 2006年3月2日 8:44 Thank you for your advice. I checked retun value of mmap(), and it is not NULL. I'll check vcpu_translate(). Sorry for not clear, we need to check if the return address of mmap is in region0, Current vcpu_translate may not handle all scenarios when fault address is in region 0. Thanks, Anthony ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Re: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
On 28 Feb 2006, at 21:45, Magenheimer, Dan (HP Labs Fort Collins) wrote: We are seeing a strange problem where a recent cset causes Red Hat to fail domU boot on ia64 complaining of a hotplug problem but doesn't cause any problem for Suse or Debian. It is likely some subtle difference (or bug), perhaps in mmap? Suggestions/advice from anyone more familiar with distro differences (on ia64) would be appreciated. Changeset is xen-unstable 8783 (Use /dev/kmem to map dom0 xenstore page instead of abusing the foreign mapping interface., Feb 8, committed by Christian). Backing out this changeset or using the small patch below causes the problem to go away, so we have a workaround, but a root cause would be nice to know and fix. Not very many apps use /dev/kmem, and xenstored only uses it once, to map domain0's xenbus page. Can't you just trace the hell out of it and find out exactly what MFN is mapping and why it (presumably) is different from the one allocated by the domain0 kernel for the purpose (the kernel virtual address of which is exported to xenstored via /proc)? We weren't delighted to end up using /dev/kmem for this purpose, but I don't think our use is an abuse of the interface (I think we're using /dev/kmem mmap() 'within spec' :-). -- Keir ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] RE: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
It is likely some subtle difference (or bug), perhaps in mmap?
As I know, in Redhat, mmap can return NULL address, but seems xen
can't handle this situation, see below code segment:
In function vcpu_translate() of vcpu.c file
else if (!region warn_region0_address) {
REGS *regs = vcpu_regs(vcpu);
unsigned long viip = PSCB(vcpu,iip);
unsigned long vipsr = PSCB(vcpu,ipsr);
unsigned long iip = regs-cr_iip;
unsigned long ipsr = regs-cr_ipsr;
printk(vcpu_translate: bad address %p, viip=%p, vipsr=%p, iip=%p,
ipsr=%p continuing\n, address, viip, vipsr, iip, ipsr);
}
warn_region0_address is turned off by default,
so maybe we can turn on warn_region0_address to see whether this is the root
cause.
Thanks,
-Anthony
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Magenheimer, Dan
(HP Labs Fort Collins)
Sent: 2006年3月1日 5:45
To: xen-devel
Cc: yo.fujita; Akio Takebe; xen-ia64-devel@lists.xensource.com
Subject: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine
Hi all --
We are seeing a strange problem where a recent cset causes
Red Hat to fail domU boot on ia64 complaining of a hotplug
problem but doesn't cause any problem for Suse or Debian.
It is likely some subtle difference (or bug), perhaps in mmap?
Suggestions/advice from anyone more familiar with distro
differences (on ia64) would be appreciated.
Changeset is xen-unstable 8783 (Use /dev/kmem to map dom0
xenstore page instead of abusing the foreign mapping interface.,
Feb 8, committed by Christian). Backing out this changeset
or using the small patch below causes the problem to go away,
so we have a workaround, but a root cause would be nice
to know and fix.
Full thread of discussion can be found here:
http://lists.xensource.com/archives/html/xen-ia64-devel/2006-02/msg00104
.html
Thanks,
Dan
-Original Message-
From: Akio Takebe [mailto:[EMAIL PROTECTED]
Sent: Tuesday, February 28, 2006 2:47 AM
To: Magenheimer, Dan (HP Labs Fort Collins); yo.fujita;
xen-ia64-devel@lists.xensource.com
Cc: Akio Takebe
Subject: RE: [Xen-ia64-devel] Weekly benchmark results [2/3rd week]
Hi, Dan
I'm still debuging, but it is very difficult...
Much advice is welcome. :-)
Now I can boot domU by using the following patch.
diff -r 6c43118bdba8 tools/xenstore/xenstored_domain.c
--- a/tools/xenstore/xenstored_domain.c Fri Feb 24 15:41:08 2006 -0700
+++ b/tools/xenstore/xenstored_domain.c Tue Feb 28 18:20:16 2006 +0900
@@ -467,6 +467,7 @@ static int dom0_init(void)
int rc, fd;
evtchn_port_t port;
unsigned long kva;
+ unsigned long mfn;
char str[20];
struct domain *dom0;
@@ -500,9 +501,16 @@ static int dom0_init(void)
if (fd == -1)
return -1;
- dom0-interface = mmap(NULL, getpagesize(),
PROT_READ|PROT_WRITE,
- MAP_SHARED, fd, kva);
- if (dom0-interface == MAP_FAILED)
+ mfn=((0x0fff kva) 14);
+/*
+dom0-interface = mmap(NULL, getpagesize(),
PROT_READ|PROT_WRITE,
+ MAP_SHARED, fd, kva);
+*/
+ dom0-interface = xc_map_foreign_range(
+ *xc_handle, 0,
+ getpagesize(), PROT_READ|PROT_WRITE, mfn);
+ if (!dom0-interface)
+// if (dom0-interface == MAP_FAILED)
goto outfd;
close(fd);
Best Regards,
Akio Takebe
Hi Akio --
Any more progress on this issue? If you are stuck,
maybe we should post the problem to xen-devel to
see if we can get help from a Red Hat person (since
the problem doesn't occur on Suse or Debian).
Thanks,
Dan
-Original Message-
From: Akio Takebe [mailto:[EMAIL PROTECTED]
Sent: Thursday, February 23, 2006 8:45 PM
To: Magenheimer, Dan (HP Labs Fort Collins); yo.fujita;
xen-ia64-devel@lists.xensource.com
Cc: Akio Takebe
Subject: RE: [Xen-ia64-devel] Weekly benchmark results [2/3rd week]
Hi, Dan and Alex
I think this issue is only on ia64.
I seem that [EMAIL PROTECTED]/char/mem.c is used on ia64,
but [EMAIL PROTECTED]/xen/char/mem.c is used on x86.
So I think pfn or kva aren't set correctly.
We tried to boot domU with revesing cset xen-ia64-ustable.8790
and it was good work.
I'm still debugging it. :-
Best Regards,
Akio Takebe
Confirmed cset xen-unstable 8783 fails while 8782 succeeds.
Perhaps there's something different about mmap on RH
vs Suse and Debian? Perhaps only on ia64?
___
Xen-devel mailing list
[EMAIL PROTECTED]
http://lists.xensource.com/xen-devel
___
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel
