Re: [XenPPC] Why no do_softirq() after external interrupt ?
On Fri, May 18, 2007 at 01:03:35AM +, HYEONSEUNG JANG wrote: > Is there anyone who can tell me the reason why do_softirq() is not > called after external interrupt handling? Or is it just a simple > missing(but important) code in XenPPC port ? Others on this list can comment about Xen/PPC's model for handling external exceptions, but please see these two emails for notes regarding the limitations of generic Xen's design with respect to Xen/PPC's model: http://lists.xensource.com/archives/html/xen-ppc-devel/2006-11/msg00144.html http://lists.xensource.com/archives/html/xen-ppc-devel/2006-11/msg00182.html ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Fix Xen-GDB Stub (smp_sen_stop, E00 on unknown command, remove printk prefix)
On Fri, May 04, 2007 at 09:26:07PM +1000, Tony Breeds wrote: > On a wider note, How close is the SMP infrastructure to allowing > smp_send_stop() to be implemented? The SMP infrastructure supports IPIs now and has been pretty heavily tested. I think you should be able to fill in that function now. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Check for NULL source pointer
While debugging the zImage problem on a JS21, I noticed that DOM0
bootargs was filled with bogus characters. I believe something like
this is necessary.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
ofd_fixup.c |5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff -r cc72b77132f9 xen/arch/powerpc/ofd_fixup.c
--- a/xen/arch/powerpc/ofd_fixup.c Wed Mar 21 18:29:49 2007 -0500
+++ b/xen/arch/powerpc/ofd_fixup.c Tue Mar 27 19:13:03 2007 -0400
@@ -276,7 +276,10 @@ static ofdn_t ofd_chosen_props(void *m,
&path[1], sizeof (path) - 1);
}
-strlcpy(bootargs, cmdline, sizeof(bootargs));
+memset(bootargs, 0, sizeof(bootargs));
+if (cmdline) {
+strlcpy(bootargs, cmdline, sizeof(bootargs));
+}
bsz = strlen(bootargs) + 1;
rm = sizeof (bootargs) - bsz;
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] please test xen-unstable
On Mon, Mar 26, 2007 at 01:19:28PM -0500, Hollis Blanchard wrote: > Hi, all the PowerPC changes are now merged into > http://xenbits.xensource.com/xen-unstable.hg . Please test this tree and > report any problems, because these bits are the ones that will go into > the 3.0.5 release which will go into Linux distributions. Note that right now you have to use a vmlinux.strip instead of a zImage as your DOM0 image. Using this compile line, I was able to get things booting fine on a JS21: make CROSS_COMPILE=powerpc64-linux- XEN_TARGET_ARCH=powerpc64 \ DOM0_IMAGE=~/devel/xen/linux-ppc-2.6.hg/vmlinux.strip Hopefully the zImage bug will be fixed before the 3.0.5 release. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Question about netbooting with initrd/initramfs
On Thu, Jan 25, 2007 at 04:39:39PM -0500, David M Daly wrote: > I've been having some trouble netbooting linux on a JS21 with an initramfs Getting a builtin initramfs to work on PPC is tricky. The only way I have found to do it is a fragile two-stage process. I assume that you want to build an initramfs necessary and sufficient to run xend and launch a domU, so in the following I describe one technique that may prove useful for that attempt. It attempts to find by inspection on the NFS server the minimal set of files required to launch a domU. In the following, NFS denotes the host from which you serve your nfsroot, and DEV denotes the machine on which you compile Linux and Xen. NFS: cd ~/devel/nfsroots/ touch TIMESTAMP DEV: cd xenppc-unstable.hg/xen net-boot -m VICTIM -f xen NFS: cd sles10-rootfs-2006-11-20/ sudo find * -anewer ../TIMESTAMP > ../xend.list cd .. sudo rsync -a --files-from=xend.list sles10-rootfs-2006-11-20/ xend/ DEV: cd ../../linux-ppc-2.6.hg/ sudo rsync -a [EMAIL PROTECTED]:/home/you/devel/nfsroots/xend/ xend/ sudo chmod o+r xend/etc/ssh/ssh_host_dsa_key \ xend/etc/ssh/ssh_host_key \ xend/etc/ssh/ssh_host_rsa_key \ xend/etc/ssh/sshd_config \ xend/sbin/blogd xend/usr/sbin/klogconsole At this point, you need to work around (what I think is) a bug in Linux's initramfs build logic or in PPC Linux's boot logic. First put this in a file called xend.list: dir /dev 755 0 0 nod /dev/console 644 0 0 c 5 1 nod /dev/loop0 644 0 0 b 7 0 dir /proc 755 0 0 dir /sys 755 0 0 dir /mnt 755 0 0 file /init xend/sbin/quickinit 755 0 0 Then edit Linux's .config to have: CONFIG_INITRAMFS_SOURCE="xend.list" Now you can resume the build process: DEV: scripts/gen_initramfs_list.sh xend >> xend.list make CROSS_COMPILE='distcc powerpc64-linux-' \ CROSS_COMPILE32=powerpc-linux- ARCH=powerpc zImage cd ../xenppc-unstable.hg/xen/ make CROSS_COMPILE=powerpc64-linux- XEN_TARGET_ARCH=powerpc64 \ DOM0_IMAGE =~/devel/xen/linux-ppc-2.6.hg/arch/powerpc/boot/zImage net-boot -m VICTIM -f xen So in theory, what we have now is a root filesystem sufficient to run the regular quickinit script that does the automounts, launches sshd, and starts xend. However, the total netboot image size is 13 MB, so xen boots properly and Linux starts fine, but hangs when unzipping the initramfs: gunzipping (0x140 <- 0x407000:0xeef729)... You will have to figure how how to trim files from xend.list that, while were accessed during our reference nfsroot boot process, are not strictly necessary to run xend and launch domUs. Alternatively, you can take a look at what is going wrong with Linux's initramfs logic in which it cannot handle an approximately 9MB compressed initramfs. Just to make sure that we are on the right track, if you place the 5-tuples at the end of this email into xend.list, the resultant initramfs will provide you with a basic libc6 environment with bash and a few standard utilities. The xen netboot image is 7.6 MB. Note that after editing xend.list and before recompiling, you have to do this to work around a problem in Linux's initramfs build deps: rm usr/initramfs_data.cpio* Good luck. I hope you are able to extend this technique, or discover a better one, to build a minimal Xen dom0. --- dir /dev 755 0 0 nod /dev/console 644 0 0 c 5 1 nod /dev/loop0 644 0 0 b 7 0 dir /proc 755 0 0 dir /sys 755 0 0 dir /mnt 755 0 0 slink /init /bin/bash 755 0 0 dir /bin 755 0 0 file /bin/bash xend/bin/bash 755 0 0 file /bin/cat xend/bin/cat 755 0 0 file /bin/cp xend/bin/cp 755 0 0 file /bin/ls xend/bin/ls 755 0 0 file /bin/mount xend/bin/mount 4755 0 0 file /bin/ps xend/bin/ps 555 0 0 file /bin/rm xend/bin/rm 755 0 0 file /bin/sync xend/bin/sync 755 0 0 file /bin/umount xend/bin/umount 4755 0 0 dir /lib 755 0 0 slink /lib/ld.so.1 ld-2.4.so 777 0 0 slink /lib/libacl.so.1 libacl.so.1.1.0 777 0 0 slink /lib/libattr.so.1 libattr.so.1.1.0 777 0 0 slink /lib/libbz2.so.1 libbz2.so.1.0.0 777 0 0 slink /lib/libcap.so.1 libcap.so.1.92 777 0 0 slink /lib/libcom_err.so.2 libcom_err.so.2.1 777 0 0 slink /lib/libcrypt.so.1 libcrypt-2.4.so 777 0 0 slink /lib/libdl.so.2 libdl-2.4.so 777 0 0 slink /lib/libgetconfig.so.1 libgetconfig.so.1.1.0 777 0 0 slink /lib/libhistory.so.5 libhistory.so.5.1 777 0 0 slink /lib/libncurses.so.5 libncurses.so.5.5 777 0 0 slink /lib/libnsl.so.1 libnsl-2.4.so 777 0 0 slink /lib/libnss_compat.so.2 libnss_compat-2.4.so 777 0 0 slink /lib/libnss_dns.so.2 libnss_dns-2.4.so 777 0 0 slink /lib/libnss_files.so.2 libnss_files-2.4.so 777 0 0 slink /lib/libnss_nis.so.2 libnss_nis-2.4.so 777 0 0 slink /lib/libpam.so.0 libpam.so.0.81.2 777 0 0 slink /lib/libreadline.so.5 libreadline.so.5.1 777 0 0 slink /lib/libresolv.so.2 libresol
Re: [XenPPC] IPI problems
On Fri, Jan 12, 2007 at 05:45:03PM -0600, Hollis Blanchard wrote:
> We seem to have an IPI problem, which causes vcpu_pause() to hang the
> system. The following patch, tested on JS20 and JS21, illustrates it.
> Before dom0 starts, IPIs work fine. After Linux's mpic_init(), IPIs (as
> triggered by the 'I' keyhandler) lock the machine. Actually, it looks
> like a message is trying to get out, because after a while we see a '('
> emitted (presumably the first character in "(XEN)").
No, this is almost certainly our code that checks that the IPI start is
acked. If you run with `sync_console' you should see periodic messages
about start stalls.
> (When I comment out mpic_init() in dom0, Xen IPIs continue to work but
> real IRQs (e.g. the IDE controller) fail in dom0.)
Make sure you did not merge this out:
http://lists.xensource.com/archives/html/xen-ppc-devel/2006-11/msg00149.html
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] Machine check: instruction-fetch TLB tablewalk
Just saw this on a JS21 blade (internal name is cso52): (XEN) MACHINE CHECK: IS Recoverable (XEN) [ Xen-3.0-unstable ] (XEN) CPU: DOMID: (XEN) pc 10046510 msr 000cf032 (XEN) lr 10063bf4 ctr 0fde93c0 (XEN) srr0 srr1 (XEN) r00: 10063be4 ffda9710 f7fcd470 0003 (XEN) r04: 100a9268 0001 0030 fefefeff (XEN) r08: 0fecb4d8 100a 0019 (XEN) r12: 28242424 100a3300 1002 (XEN) r16: 100a 100a9008 100a 1008 (XEN) r20: 1006 (XEN) r24: 0008 100a9df8 100a (XEN) r28: 100a9268 100a 10063bc0 (XEN) dar 0xffda9628, dsisr 0x0220 (XEN) hid4 0x (XEN) ---[ backtrace ]--- (XEN) SP (ffda9710) is not in xen space (XEN) SRR1: 0x000cf032 (XEN) 0b11: Exception caused by a hardware uncorrectable (XEN) error (UE) detected while doing a reload of an (XEN) instruction-fetch TLB tablewalk. (XEN) (XEN) DSISR: 0x0220 (XEN) program_exception: machine check (XEN) machine_halt called: spinning (XEN) machine_halt called ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Fix systemsim-gpul failure to boot
On Sat, Dec 09, 2006 at 11:05:31PM -0500, Amos Waterland wrote: > This patch fixes the fencepost error and adds a check for the failure to > initialize the allocator, which would have make Mark's FTB much easier > to debug. This works for me on the public systemsim-gpul release. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix systemsim-gpul failure to boot
At present, Xen/PPC fails to boot on the systemsim-gpul public release,
as first reported here by Mark Mergen. The symptom of the failure is
a "Could not allocate RTAS tree" panic.
The problem turned out to be that the first interrupt vector page was
being handed out by the allocator, and one of the checks along the RTAS
allocate path happened to interpret the base address of 0 as a failure
return code. The core problem actually occurred much earlier, and was
that the logic that marks the first four pages as being off limits to
the allocator was never being run because there was an off-by-one error
in the code that parses the "reg" property of the "memory" OF node.
This patch fixes the fencepost error and adds a check for the failure to
initialize the allocator, which would have make Mark's FTB much easier
to debug.
---
boot_of.c |7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff -r 7669fca80bfc xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cMon Dec 04 11:46:53 2006 -0500
+++ b/xen/arch/powerpc/boot_of.cSat Dec 09 22:49:14 2006 -0500
@@ -608,7 +608,7 @@ static ulong boot_of_mem_init(void)
if (size_cells == 2 && (r < l) )
size = (size << 32) | reg[r++];
-if (r >= l)
+if (r > l)
break; /* partial line. Skip */
boot_of_alloc_init(p, addr_cells, size_cells);
@@ -1290,6 +1290,7 @@ multiboot_info_t __init *boot_of_init(
{
static multiboot_info_t mbi;
void *oft;
+int r;
of_vec = vec;
of_msr = orig_msr;
@@ -1316,7 +1317,9 @@ multiboot_info_t __init *boot_of_init(
of_printf("%s: _start %p _end %p 0x%lx\n", __func__, _start, _end, r6);
boot_of_fix_maple();
-boot_of_mem_init();
+r = boot_of_mem_init();
+if (r == 0)
+of_panic("failure to initialize memory allocator");
boot_of_bootargs(&mbi);
oft = boot_of_module(r3, r4, &mbi);
boot_of_cpus();
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Disable DPM until code is audited
On Sat, Dec 02, 2006 at 10:31:11AM -0500, Jimi Xenidis wrote: > The following patch results in SMP stability on Maple. Amos, > Kawachiya-san, could one of you ack it with the JS20 in question? > > @@ -193,10 +193,10 @@ void cpu_initialize(int cpuid) > mtdec(timebase_freq); > mthdec(timebase_freq); > -/* FIXME Do not set the NAP and DPM bits in HID0 until we have had a > - * chance to audit the safe halt and idle loop code. */ > +/* FIXME Do not set the NAP bit in HID0 until we have had a chance > + * to audit the safe halt and idle loop code. */ > hid0.bits.nap = 0; /* NAP */ > -hid0.bits.dpm = 0; /* Dynamic Power Management */ > +hid0.bits.dpm = 1; /* Dynamic Power Management */ > hid0.bits.nhr = 1; /* Not Hard Reset */ > hid0.bits.hdice_en = 1; /* enable HDEC */ This works on the JS20 in TRL. Acked-by: Amos Waterland <[EMAIL PROTECTED]> ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Disable DPM until code is audited
Do not set the NAP and DPM bits in HID0 until we have had a chance to audit the safe halt and idle loop code. Not setting these bits allows the model 884241X JS20 blade in TRL to boot correctly, and possibly also the Maple in YKT. Thanks to Jimi for his help in this matter. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- ppc970.c |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff -r 927f25df5154 xen/arch/powerpc/powerpc64/ppc970.c --- a/xen/arch/powerpc/powerpc64/ppc970.c Thu Nov 30 16:21:22 2006 -0500 +++ b/xen/arch/powerpc/powerpc64/ppc970.c Fri Dec 01 16:24:32 2006 -0500 @@ -193,8 +193,8 @@ void cpu_initialize(int cpuid) mtdec(timebase_freq); mthdec(timebase_freq); -hid0.bits.nap = 1; /* NAP */ -hid0.bits.dpm = 1; /* Dynamic Power Management */ +hid0.bits.nap = 0; /* NAP */ +hid0.bits.dpm = 0; /* Dynamic Power Management */ hid0.bits.nhr = 1; /* Not Hard Reset */ hid0.bits.hdice_en = 1; /* enable HDEC */ hid0.bits.en_therm = 0; /* ! Enable ext thermal ints */ ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Re: FYI: [Xen-devel] FW: Request for Xen console device (xvc0)
Redhat took my patch and rolled it into their paravirtualized framebuffer effort, so I am going to wait a few days to see if that goes in. On Wed, Nov 29, 2006 at 07:24:07AM -0500, Jimi Xenidis wrote: > Amos, you see this? > > Begin forwarded message: > > >From: Keir Fraser <[EMAIL PROTECTED]> > >Date: November 29, 2006 6:31:34 AM EST > >To: Xen-devel <[EMAIL PROTECTED]> > >Subject: [Xen-devel] FW: Request for Xen console device (xvc0) > > > > Lanana has provided an xvc0 allocation, so I'll apply a patch to add > > this as an option to the console driver. > > > > -- Keir > > > >-- Forwarded Message > >From: "Mathiasen, Torben" <[EMAIL PROTECTED]> > >Date: Wed, 29 Nov 2006 11:58:02 +0100 > >To: Keir Fraser <[EMAIL PROTECTED]> > >Conversation: Request for Xen console device (xvc0) > >Subject: RE: Request for Xen console device (xvc0) > > > > > Please can we be allocated a major/minor for the following new > > > device: > > >/dev/xvc0Xen virtual console - port 0 > > > > > > > This has been added to lanana.org with major/minor 204/191. > > > > Please verify the data is correct. > > > > Thanks, > > Torben > > > >-- End of Forwarded Message ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: SMP issue (Re: Status summary Was: [XenPPC] Cannot boot from local disk)
On Wed, Nov 29, 2006 at 03:14:54PM +0900, Kiyokuni Kawachiya wrote: > > I can confirm that the JS20 of Kawachiya-san does not boot current > > Xen/PPC without the nosmp option. I got some time on the blade and > > narrowed the point of failure down to somewhere in the synchronize_rcu > > call that synchronize_net makes in dom0 Linux init. I will work on > > it tomorrow during JST night. > > Thank you for the investigation. > > Since no one complains about this, the problem (XenPPC does not boot with > SMP) seems to be my local problem. > I am afraid that I did some trivial mistake in setting up my JS20 blade. I do not think you did anything wrong in setting up the blade. It is a JS20 model that Xen/PPC has not seen before and we will have to find out what is going wrong. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: SMP issue (Re: Status summary Was: [XenPPC] Cannot boot from local disk)
I can confirm that the JS20 of Kawachiya-san does not boot current Xen/PPC without the nosmp option. I got some time on the blade and narrowed the point of failure down to somewhere in the synchronize_rcu call that synchronize_net makes in dom0 Linux init. I will work on it tomorrow during JST night. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix IPI stall timeout without using timebase_freq
When using the register dump feature of Xen, one will sometimes see a
message about an IPI finish stall. This is because of an int to long
comparison bug, so fix it by doing proper nanosecond based time
accounting with no explicit reference to timebase_freq.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
smp.c | 39 +--
1 file changed, 25 insertions(+), 14 deletions(-)
diff -r e01e08ca629b xen/arch/powerpc/smp.c
--- a/xen/arch/powerpc/smp.cTue Nov 28 17:01:00 2006 -0500
+++ b/xen/arch/powerpc/smp.cTue Nov 28 18:28:35 2006 -0500
@@ -90,7 +90,8 @@ int on_selected_cpus(
int retry,
int wait)
{
-int t, retval = 0, nr_cpus = cpus_weight(selected);
+int retval = 0, nr_cpus = cpus_weight(selected);
+unsigned long start, stall = SECONDS(1);
spin_lock(&call_lock);
@@ -104,19 +105,21 @@ int on_selected_cpus(
send_IPI_mask(selected, CALL_FUNCTION_VECTOR);
/* We always wait for an initiation ACK from remote CPU. */
-for (t = 0; atomic_read(&call_data.started) != nr_cpus; t++) {
-if (t && t % timebase_freq == 0) {
+for (start = NOW(); atomic_read(&call_data.started) != nr_cpus; ) {
+if (NOW() > start + stall) {
printk("IPI start stall: %d ACKS to %d SYNS\n",
atomic_read(&call_data.started), nr_cpus);
+ start = NOW();
}
}
/* If told to, we wait for a completion ACK from remote CPU. */
if (wait) {
-for (t = 0; atomic_read(&call_data.finished) != nr_cpus; t++) {
-if (t > timebase_freq && t % timebase_freq == 0) {
+for (start = NOW(); atomic_read(&call_data.finished) != nr_cpus; ) {
+if (NOW() > start + stall) {
printk("IPI finish stall: %d ACKS to %d SYNS\n",
atomic_read(&call_data.finished), nr_cpus);
+start = NOW();
}
}
}
@@ -168,6 +171,11 @@ void smp_message_recv(int msg, struct cp
#ifdef DEBUG_IPI
static void debug_ipi_ack(void *info)
{
+if (info) {
+ unsigned long start, stall = SECONDS(5);
+ for (start = NOW(); NOW() < start + stall; );
+ printk("IPI recv on cpu #%d: %s\n", smp_processor_id(), (char *)info);
+}
return;
}
@@ -175,12 +183,12 @@ void ipi_torture_test(void)
{
int cpu;
unsigned long before, after, delta;
-unsigned long min = ~0, max = 0, mean = 0, sum = 0, tick = 0;
+unsigned long min = ~0, max = 0, mean = 0, sum = 0, trials = 0;
cpumask_t mask;
cpus_clear(mask);
-while (tick < 100) {
+while (trials < 100) {
for_each_online_cpu(cpu) {
cpu_set(cpu, mask);
before = mftb();
@@ -192,12 +200,15 @@ void ipi_torture_test(void)
if (delta > max) max = delta;
if (delta < min) min = delta;
sum += delta;
-tick++;
-}
-}
-
-mean = sum / tick;
-
-printk("IPI tb ticks: min = %ld max = %ld mean = %ld\n", min, max, mean);
+trials++;
+}
+}
+
+mean = tb_to_ns(sum / trials);
+
+printk("IPI latency: min = %ld ticks, max = %ld ticks, mean = %ldns\n",
+ min, max, mean);
+
+smp_call_function(debug_ipi_ack, "Hi", 0, 1);
}
#endif
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix IPI stall timeout
When using the register dump feature of Xen, one will sometimes see a
message about an IPI finish stall. This is because of an int to long
comparison bug, so fix it by doing proper nanosecond based time accounting.
As a side note, our IPI remote function call latency of completion on
a JS21 blade is: min = 34 ticks, max = 119 ticks, mean = 2691ns.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
smp.c | 39 +--
1 file changed, 25 insertions(+), 14 deletions(-)
diff -r e01e08ca629b xen/arch/powerpc/smp.c
--- a/xen/arch/powerpc/smp.cTue Nov 28 17:01:00 2006 -0500
+++ b/xen/arch/powerpc/smp.cTue Nov 28 17:40:50 2006 -0500
@@ -90,7 +90,8 @@ int on_selected_cpus(
int retry,
int wait)
{
-int t, retval = 0, nr_cpus = cpus_weight(selected);
+int retval = 0, nr_cpus = cpus_weight(selected);
+unsigned long start, stall = tb_to_ns(timebase_freq);
spin_lock(&call_lock);
@@ -104,19 +105,21 @@ int on_selected_cpus(
send_IPI_mask(selected, CALL_FUNCTION_VECTOR);
/* We always wait for an initiation ACK from remote CPU. */
-for (t = 0; atomic_read(&call_data.started) != nr_cpus; t++) {
-if (t && t % timebase_freq == 0) {
+for (start = NOW(); atomic_read(&call_data.started) != nr_cpus; ) {
+if (NOW() > start + stall) {
printk("IPI start stall: %d ACKS to %d SYNS\n",
atomic_read(&call_data.started), nr_cpus);
+ start = NOW();
}
}
/* If told to, we wait for a completion ACK from remote CPU. */
if (wait) {
-for (t = 0; atomic_read(&call_data.finished) != nr_cpus; t++) {
-if (t > timebase_freq && t % timebase_freq == 0) {
+for (start = NOW(); atomic_read(&call_data.finished) != nr_cpus; ) {
+if (NOW() > start + stall) {
printk("IPI finish stall: %d ACKS to %d SYNS\n",
atomic_read(&call_data.finished), nr_cpus);
+start = NOW();
}
}
}
@@ -168,6 +171,11 @@ void smp_message_recv(int msg, struct cp
#ifdef DEBUG_IPI
static void debug_ipi_ack(void *info)
{
+if (info) {
+ unsigned long start, stall = tb_to_ns(5 * timebase_freq);
+ for (start = NOW(); NOW() < start + stall; );
+ printk("IPI recv on cpu #%d: %s\n", smp_processor_id(), (char *)info);
+}
return;
}
@@ -175,12 +183,12 @@ void ipi_torture_test(void)
{
int cpu;
unsigned long before, after, delta;
-unsigned long min = ~0, max = 0, mean = 0, sum = 0, tick = 0;
+unsigned long min = ~0, max = 0, mean = 0, sum = 0, trials = 0;
cpumask_t mask;
cpus_clear(mask);
-while (tick < 100) {
+while (trials < 100) {
for_each_online_cpu(cpu) {
cpu_set(cpu, mask);
before = mftb();
@@ -192,12 +200,15 @@ void ipi_torture_test(void)
if (delta > max) max = delta;
if (delta < min) min = delta;
sum += delta;
-tick++;
-}
-}
-
-mean = sum / tick;
-
-printk("IPI tb ticks: min = %ld max = %ld mean = %ld\n", min, max, mean);
+trials++;
+}
+}
+
+mean = tb_to_ns(sum / trials);
+
+printk("IPI latency: min = %ld ticks, max = %ld ticks, mean = %ldns\n",
+ min, max, mean);
+
+smp_call_function(debug_ipi_ack, "Hi", 0, 1);
}
#endif
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Do not override smp function call wait flag
Do not override the caller's wishes regarding waiting for smp function call completion. I was being too conservative in this respect: the lock protects the call_data structure, and the function called is expected to be threadsafe. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- smp.c |1 - 1 file changed, 1 deletion(-) diff -r a8e67a19c325 xen/arch/powerpc/smp.c --- a/xen/arch/powerpc/smp.cTue Nov 28 10:33:53 2006 -0500 +++ b/xen/arch/powerpc/smp.cTue Nov 28 16:28:10 2006 -0500 @@ -97,7 +97,6 @@ int on_selected_cpus( call_data.func = func; call_data.info = info; call_data.wait = wait; -call_data.wait = 1; /* Until we get RCU around call_data. */ atomic_set(&call_data.started, 0); atomic_set(&call_data.finished, 0); mb(); ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Generalized parallel IPI handlers
Our problem with x86 do_IRQ is that it does not respect IRQ_PER_CPU, so
make our logic reflect that generically. We remove the spin lock in
this case, since the programming convention is to update irq descs
atomically (if at all). This patch allows read_clocks to work properly.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/external.c |6 ++
arch/powerpc/mpic_init.c |6 --
include/asm-powerpc/smp.h |1 -
3 files changed, 2 insertions(+), 11 deletions(-)
diff -r a8e67a19c325 xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Tue Nov 28 10:33:53 2006 -0500
+++ b/xen/arch/powerpc/external.c Tue Nov 28 16:14:27 2006 -0500
@@ -82,15 +82,13 @@ void do_external(struct cpu_user_regs *r
vec = xen_mpic_get_irq(regs);
-if (vector_is_ipi(vec)) {
-/* do_IRQ is fundamentally broken for reliable IPI delivery. */
+if (irq_desc[vec].status & IRQ_PER_CPU) {
+ /* x86 do_IRQ does not respect the per cpu flag. */
irq_desc_t *desc = &irq_desc[vec];
regs->entry_vector = vec;
-spin_lock(&desc->lock);
desc->handler->ack(vec);
desc->action->handler(vector_to_irq(vec), desc->action->dev_id, regs);
desc->handler->end(vec);
-spin_unlock(&desc->lock);
} else if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
regs->entry_vector = vec;
diff -r a8e67a19c325 xen/arch/powerpc/mpic_init.c
--- a/xen/arch/powerpc/mpic_init.c Tue Nov 28 10:33:53 2006 -0500
+++ b/xen/arch/powerpc/mpic_init.c Tue Nov 28 16:14:15 2006 -0500
@@ -448,9 +448,3 @@ int xen_mpic_get_irq(struct cpu_user_reg
return mpic_get_one_irq(mpic, regs);
}
-
-int vector_is_ipi(int vector)
-{
-BUG_ON(!mpic);
-return (mpic->ipi_offset <= vector) && (vector < mpic->ipi_offset + 4);
-}
diff -r a8e67a19c325 xen/include/asm-powerpc/smp.h
--- a/xen/include/asm-powerpc/smp.h Tue Nov 28 10:33:53 2006 -0500
+++ b/xen/include/asm-powerpc/smp.h Tue Nov 28 16:14:19 2006 -0500
@@ -51,7 +51,6 @@ void smp_call_function_interrupt(struct
void smp_call_function_interrupt(struct cpu_user_regs *regs);
void smp_event_check_interrupt(void);
void send_IPI_mask(cpumask_t mask, int vector);
-int vector_is_ipi(int vector);
#undef DEBUG_IPI
#ifdef DEBUG_IPI
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Relax IPI stall timeout
When using the register dump feature of Xen, one will often see a
message about an IPI finish stall. This is because we expected IPI
handlers to run very quickly, but in this case, the handler is doing a
lot of console I/O in order to print the register contents. So relax
the stall timeout.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
smp.c |5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff -r a8e67a19c325 xen/arch/powerpc/smp.c
--- a/xen/arch/powerpc/smp.cTue Nov 28 10:33:53 2006 -0500
+++ b/xen/arch/powerpc/smp.cTue Nov 28 13:29:50 2006 -0500
@@ -91,6 +91,7 @@ int on_selected_cpus(
int wait)
{
int t, retval = 0, nr_cpus = cpus_weight(selected);
+int stall = timebase_freq * 10;
spin_lock(&call_lock);
@@ -106,7 +107,7 @@ int on_selected_cpus(
/* We always wait for an initiation ACK from remote CPU. */
for (t = 0; atomic_read(&call_data.started) != nr_cpus; t++) {
-if (t && t % timebase_freq == 0) {
+if (t > 0 && t % stall == 0) {
printk("IPI start stall: %d ACKS to %d SYNS\n",
atomic_read(&call_data.started), nr_cpus);
}
@@ -115,7 +116,7 @@ int on_selected_cpus(
/* If told to, we wait for a completion ACK from remote CPU. */
if (wait) {
for (t = 0; atomic_read(&call_data.finished) != nr_cpus; t++) {
-if (t > timebase_freq && t % timebase_freq == 0) {
+if (t > 0 && t % stall == 0) {
printk("IPI finish stall: %d ACKS to %d SYNS\n",
atomic_read(&call_data.finished), nr_cpus);
}
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Make IPI handlers run in parallel
Make a copy of the vector information while protected by a lock, then
release the lock and use that information to acknowledge and act on the IPI.
We always have a consistent set of information in this case, as opposed
to the case of dropping and reaquiring the lock.
This patch makes read_clocks work properly.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
external.c | 18 +++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff -r a8e67a19c325 xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Tue Nov 28 10:33:53 2006 -0500
+++ b/xen/arch/powerpc/external.c Tue Nov 28 13:28:18 2006 -0500
@@ -84,13 +84,25 @@ void do_external(struct cpu_user_regs *r
if (vector_is_ipi(vec)) {
/* do_IRQ is fundamentally broken for reliable IPI delivery. */
+void (*ack)(unsigned int irq);
+void (*handler)(int, void *, struct cpu_user_regs *);
+void (*end)(unsigned int irq);
+int irq;
+void *dev_id;
irq_desc_t *desc = &irq_desc[vec];
regs->entry_vector = vec;
+
spin_lock(&desc->lock);
-desc->handler->ack(vec);
-desc->action->handler(vector_to_irq(vec), desc->action->dev_id, regs);
-desc->handler->end(vec);
+ack = desc->handler->ack;
+handler = desc->action->handler;
+end = desc->handler->end;
+irq = vector_to_irq(vec);
+dev_id = desc->action->dev_id;
spin_unlock(&desc->lock);
+
+ack(vec);
+handler(irq, dev_id, regs);
+end(vec);
} else if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
regs->entry_vector = vec;
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [xenppc-unstable] [XEN][POWERPC] SMP/IPI/MB combined
On Tue, Nov 28, 2006 at 10:47:31AM -0500, Jimi Xenidis wrote: > On Nov 28, 2006, at 1:37 AM, Amos Waterland wrote: > > The compiler is not told that read_clocks_cpumask is volatile, so it > > is free to turn that loop into an infinite loop, as my gcc 4.1.1 > > cross-compiler does. I am surprised that other Xen architectures > > have not seen the same problem. > > Found it, cpu_relax() is supposed to contain barrier() call. > My fault but it coulda been hollis :) Excellent, thanks. > you _always_ have to wait for call_data.started it means that its > safe to reuse call_data. Yes, I know :) I only did that as a crude approximation, the proper solution (pending your design comments), is not serializing do_external with a lock (see the next patch I sent). ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] multicast function invocations
To summarize the situation, I found two problems.
1. Core Xen has a bug (I believe) in which they do not mark their cpu
mask volatile, so the compiler generates an infinite loop in read_clocks.
I will send some patches upstream to resolve this issue.
2. Xen/PPC has a problem in that its IPI callbacks (remote function
invocations) do not actually happen in parallel, which breaks the
design of read_clocks. Our IPI callbacks are serialized by the
design we copied from Xen/x86, which is to acquire a per-vector lock
very early in the EE handling path (see do_external).
I guess my real question is: will Xen/PPC ever in the future run its
IPI remote function callbacks with EE enabled? If the plan is to
keep things the way they are now, then we should remove the
per-vector lock entirely.
The following is a patch that implements the above two conclusions and
which allows 'C-aC-aC-at' to work properly. Comments?
---
arch/powerpc/external.c |2 --
common/keyhandler.c |2 +-
include/xen/cpumask.h |2 +-
3 files changed, 2 insertions(+), 4 deletions(-)
diff -r 305751a5281e xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Wed Nov 22 16:29:25 2006 -0500
+++ b/xen/arch/powerpc/external.c Tue Nov 28 03:07:10 2006 -0500
@@ -86,11 +86,9 @@ void do_external(struct cpu_user_regs *r
/* do_IRQ is fundamentally broken for reliable IPI delivery. */
irq_desc_t *desc = &irq_desc[vec];
regs->entry_vector = vec;
-spin_lock(&desc->lock);
desc->handler->ack(vec);
desc->action->handler(vector_to_irq(vec), desc->action->dev_id, regs);
desc->handler->end(vec);
-spin_unlock(&desc->lock);
} else if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
regs->entry_vector = vec;
diff -r 305751a5281e xen/common/keyhandler.c
--- a/xen/common/keyhandler.c Wed Nov 22 16:29:25 2006 -0500
+++ b/xen/common/keyhandler.c Tue Nov 28 03:06:24 2006 -0500
@@ -193,7 +193,7 @@ static void dump_domains(unsigned char k
read_unlock(&domlist_lock);
}
-static cpumask_t read_clocks_cpumask = CPU_MASK_NONE;
+static cpumask_t volatile read_clocks_cpumask = CPU_MASK_NONE;
static s_time_t read_clocks_time[NR_CPUS];
static void read_clocks_slave(void *unused)
diff -r 305751a5281e xen/include/xen/cpumask.h
--- a/xen/include/xen/cpumask.h Wed Nov 22 16:29:25 2006 -0500
+++ b/xen/include/xen/cpumask.h Tue Nov 28 03:06:24 2006 -0500
@@ -177,7 +177,7 @@ static inline int __cpus_subset(const cp
}
#define cpus_empty(src) __cpus_empty(&(src), NR_CPUS)
-static inline int __cpus_empty(const cpumask_t *srcp, int nbits)
+static inline int __cpus_empty(const cpumask_t volatile *srcp, int nbits)
{
return bitmap_empty(srcp->bits, nbits);
}
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Crash on js12
On Mon, Nov 27, 2006 at 12:04:36PM -0500, Maria Butrico wrote: > This problem is solved on the latest version of Xen. I do not think the problem is completely solved. I am still seeing repeated prints like this: (XEN) CPU[PIR:1 IPI:1 Logical:1] Hello World!CPU[PIR:1 IPI:1 Logical:1] Hello World! (XEN) :1] Hello World!Got ack ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [xenppc-unstable] [XEN][POWERPC] SMP/IPI/MB combined
This will have to be reworked and broken up into individual parts for
submission, but here is what is necessary to make 'C-a C-a C-a t' work
properly.
Jimi, when you disassemble xen-syms compiled without this patch, do you
see a bogus infinite loop in read_clocks? The compiler is not told that
read_clocks_cpumask is volatile, so it is free to turn that loop into an
infinite loop, as my gcc 4.1.1 cross-compiler does. I am surprised that
other Xen architectures have not seen the same problem.
arch/powerpc/smp.c| 18 ++
common/keyhandler.c |2 +-
include/xen/cpumask.h |2 +-
3 files changed, 12 insertions(+), 10 deletions(-)
diff -r 305751a5281e xen/arch/powerpc/smp.c
--- a/xen/arch/powerpc/smp.cWed Nov 22 16:29:25 2006 -0500
+++ b/xen/arch/powerpc/smp.cTue Nov 28 00:45:24 2006 -0500
@@ -91,31 +91,33 @@ int on_selected_cpus(
int wait)
{
int t, retval = 0, nr_cpus = cpus_weight(selected);
+int stall = timebase_freq * 10;
spin_lock(&call_lock);
call_data.func = func;
call_data.info = info;
call_data.wait = wait;
-call_data.wait = 1; /* Until we get RCU around call_data. */
atomic_set(&call_data.started, 0);
atomic_set(&call_data.finished, 0);
mb();
send_IPI_mask(selected, CALL_FUNCTION_VECTOR);
-/* We always wait for an initiation ACK from remote CPU. */
-for (t = 0; atomic_read(&call_data.started) != nr_cpus; t++) {
-if (t && t % timebase_freq == 0) {
-printk("IPI start stall: %d ACKS to %d SYNS\n",
- atomic_read(&call_data.started), nr_cpus);
-}
+/* If told to, we wait for an initiation ACK from remote CPU. */
+if (wait) {
+ for (t = 0; atomic_read(&call_data.started) != nr_cpus; t++) {
+ if (t > 0 && t % stall == 0) {
+ printk("IPI start stall: %d ACKS to %d SYNS\n",
+ atomic_read(&call_data.started), nr_cpus);
+ }
+ }
}
/* If told to, we wait for a completion ACK from remote CPU. */
if (wait) {
for (t = 0; atomic_read(&call_data.finished) != nr_cpus; t++) {
-if (t > timebase_freq && t % timebase_freq == 0) {
+if (t > 0 && t % stall == 0) {
printk("IPI finish stall: %d ACKS to %d SYNS\n",
atomic_read(&call_data.finished), nr_cpus);
}
diff -r 305751a5281e xen/common/keyhandler.c
--- a/xen/common/keyhandler.c Wed Nov 22 16:29:25 2006 -0500
+++ b/xen/common/keyhandler.c Tue Nov 28 00:12:14 2006 -0500
@@ -193,7 +193,7 @@ static void dump_domains(unsigned char k
read_unlock(&domlist_lock);
}
-static cpumask_t read_clocks_cpumask = CPU_MASK_NONE;
+static cpumask_t volatile read_clocks_cpumask = CPU_MASK_NONE;
static s_time_t read_clocks_time[NR_CPUS];
static void read_clocks_slave(void *unused)
diff -r 305751a5281e xen/include/xen/cpumask.h
--- a/xen/include/xen/cpumask.h Wed Nov 22 16:29:25 2006 -0500
+++ b/xen/include/xen/cpumask.h Tue Nov 28 00:12:42 2006 -0500
@@ -177,7 +177,7 @@ static inline int __cpus_subset(const cp
}
#define cpus_empty(src) __cpus_empty(&(src), NR_CPUS)
-static inline int __cpus_empty(const cpumask_t *srcp, int nbits)
+static inline int __cpus_empty(const cpumask_t volatile *srcp, int nbits)
{
return bitmap_empty(srcp->bits, nbits);
}
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] SMP/IPI/MB combined
This patch rolls up and rebases the following patches for submission
against current tip of tree:
* Memory barrier after SP store
* IPI support
* SMP support
The only changes from the previous submission other than trivial
fast-forward merges are to remove the ERAT flush, since it was comitted
seperately, and to make the status messages about waiting for remote
function completion ACKs only kick in after a full second has passed.
Note that this path REQUIRES that some form of the dom0 Linux patch
titled "Make Linux bail out of IPI vector reset" be applied.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/external.c| 32 +
arch/powerpc/mpic.c|9 -
arch/powerpc/mpic_init.c | 48
arch/powerpc/setup.c | 50 +
arch/powerpc/smp.c | 135 +++--
include/asm-powerpc/mach-default/irq_vectors.h | 22
include/asm-powerpc/smp.h | 16 ++
7 files changed, 256 insertions(+), 56 deletions(-)
diff -r 79015565d348 xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Tue Nov 21 15:25:02 2006 -0600
+++ b/xen/arch/powerpc/external.c Tue Nov 21 20:41:27 2006 -0500
@@ -82,7 +82,16 @@ void do_external(struct cpu_user_regs *r
vec = xen_mpic_get_irq(regs);
-if (vec != -1) {
+if (vector_is_ipi(vec)) {
+ /* do_IRQ is fundamentally broken for reliable IPI delivery. */
+ irq_desc_t *desc = &irq_desc[vec];
+ regs->entry_vector = vec;
+ spin_lock(&desc->lock);
+ desc->handler->ack(vec);
+ desc->action->handler(vector_to_irq(vec), desc->action->dev_id, regs);
+ desc->handler->end(vec);
+ spin_unlock(&desc->lock);
+} else if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
regs->entry_vector = vec;
do_IRQ(regs);
@@ -253,3 +262,24 @@ int ioapic_guest_write(unsigned long phy
BUG_ON(val != val);
return 0;
}
+
+void send_IPI_mask(cpumask_t mask, int vector)
+{
+unsigned int cpus;
+int const bits = 8 * sizeof(cpus);
+
+switch(vector) {
+case CALL_FUNCTION_VECTOR:
+case EVENT_CHECK_VECTOR:
+ break;
+default:
+ BUG();
+ return;
+}
+
+BUG_ON(NR_CPUS > bits);
+BUG_ON(fls(mask.bits[0]) > bits);
+
+cpus = mask.bits[0];
+mpic_send_ipi(vector, cpus);
+}
diff -r 79015565d348 xen/arch/powerpc/mpic.c
--- a/xen/arch/powerpc/mpic.c Tue Nov 21 15:25:02 2006 -0600
+++ b/xen/arch/powerpc/mpic.c Tue Nov 21 20:41:27 2006 -0500
@@ -27,10 +27,6 @@
#define alloc_bootmem(x) xmalloc_bytes(x)
-#define request_irq(irq, handler, f, devname, dev_id) \
-panic("IPI requested: %d: %p: %s: %p\n", irq, handler, devname, dev_id)
-
-typedef int irqreturn_t;
#define IRQ_NONE (0)
#define IRQ_HANDLED(1)
@@ -96,11 +92,6 @@ typedef int irqreturn_t;
#endif
#include
#include
-
-static inline void smp_message_recv(int msg, struct pt_regs *regs)
-{
-return;
-}
#ifdef DEBUG
#define DBG(fmt...) printk(fmt)
diff -r 79015565d348 xen/arch/powerpc/mpic_init.c
--- a/xen/arch/powerpc/mpic_init.c Tue Nov 21 15:25:02 2006 -0600
+++ b/xen/arch/powerpc/mpic_init.c Tue Nov 21 20:41:27 2006 -0500
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "mpic_init.h"
#include "oftree.h"
#include "of-devtree.h"
@@ -358,6 +359,42 @@ static struct hw_interrupt_type *share_m
#endif
+static unsigned int mpic_startup_ipi(unsigned int irq)
+{
+mpic->hc_ipi.enable(irq);
+return 0;
+}
+
+int request_irq(unsigned int irq,
+ irqreturn_t (*handler)(int, void *, struct cpu_user_regs *),
+ unsigned long irqflags, const char * devname, void *dev_id)
+{
+int retval;
+struct irqaction *action;
+void (*func)(int, void *, struct cpu_user_regs *);
+
+action = xmalloc(struct irqaction);
+if (!action) {
+ BUG();
+ return -ENOMEM;
+}
+
+/* Xen's handler prototype is slightly different than Linux's. */
+func = (void (*)(int, void *, struct cpu_user_regs *))handler;
+
+action->handler = func;
+action->name = devname;
+action->dev_id = dev_id;
+
+retval = setup_irq(irq, action);
+if (retval) {
+ BUG();
+ xfree(action);
+}
+
+return retval;
+}
+
struct hw_interrupt_type *xen_mpic_init(struct hw_interrupt_type *xen_irq)
{
unsigned int isu_size;
@@ -397,6 +434,11 @@ struct hw_interrupt_type *xen_mpic_init(
hit = share_mpic(&mpic->hc_irq, xen_irq);
printk("%s: success\n", __func__);
+
+mpic->hc_ipi.ack = xen_irq->ack;
+mpic->hc_ipi.startup = mpic_startup_ipi;
+mpic_request_ipis();
+
return hit;
}
@@ -406
[XenPPC] [PATCH] Make Linux bail out of IPI vector reset
Xen/PPC currently shares the mpic with dom0 Linux. In order for
interprocessor interrupts to work, we cannot allow Linux to reset the
mpic (a patch for the latter was already merged) nor to reset the IPI
vectors and change their priority.
This patch introduces a feature flag, rather than trying to code a new
function called xen_mpic_init and duplicating all the associated
machinery it pulls in. I tried the first approach, and it results in a
lot of code duplication, which is a worse maintenence burden than this,
I believe. Jimi may not like this approach, but I believe it is the
best solution until we completely virtualize the PIC at some point in
the future.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/platforms/xen/evtchn.c |1 +
arch/powerpc/sysdev/mpic.c |4
include/asm-powerpc/mpic.h |2 ++
3 files changed, 7 insertions(+)
diff -r 4e5b95219939 arch/powerpc/platforms/xen/evtchn.c
--- a/arch/powerpc/platforms/xen/evtchn.c Mon Nov 20 12:35:47 2006 -0500
+++ b/arch/powerpc/platforms/xen/evtchn.c Tue Nov 21 20:53:20 2006 -0500
@@ -101,6 +101,7 @@ static __init struct mpic *xen_share_map
mpic = mpic_alloc(opic_addr,
MPIC_PRIMARY | MPIC_BIG_ENDIAN |
+ MPIC_SKIP_IPI_INIT |
MPIC_BROKEN_U3 /* | MPIC_WANTS_RESET */,
0, 0, 128, 128, senses, 128, "Xen-U3-MPIC");
BUG_ON(mpic == NULL);
diff -r 4e5b95219939 arch/powerpc/sysdev/mpic.c
--- a/arch/powerpc/sysdev/mpic.cMon Nov 20 12:35:47 2006 -0500
+++ b/arch/powerpc/sysdev/mpic.cTue Nov 21 21:08:32 2006 -0500
@@ -741,6 +741,9 @@ void __init mpic_init(struct mpic *mpic)
(MPIC_VEC_TIMER_0 + i));
}
+if (mpic->flags & MPIC_SKIP_IPI_INIT)
+goto ipi_bailout;
+
/* Initialize IPIs to our reserved vectors and mark them disabled for
now */
mpic_test_broken_ipi(mpic);
for (i = 0; i < 4; i++) {
@@ -756,6 +759,7 @@ void __init mpic_init(struct mpic *mpic)
#endif /* CONFIG_SMP */
}
+ ipi_bailout:
/* Initialize interrupt sources */
if (mpic->irq_count == 0)
mpic->irq_count = mpic->num_sources;
diff -r 4e5b95219939 include/asm-powerpc/mpic.h
--- a/include/asm-powerpc/mpic.hMon Nov 20 12:35:47 2006 -0500
+++ b/include/asm-powerpc/mpic.hTue Nov 21 20:53:20 2006 -0500
@@ -190,6 +190,8 @@ struct mpic
#define MPIC_BROKEN_IPI0x0008
/* MPIC wants a reset */
#define MPIC_WANTS_RESET 0x0010
+/* Skip reset of IPI vectors during init */
+#define MPIC_SKIP_IPI_INIT 0x0020
/* Allocate the controller structure and setup the linux irq descs
* for the range if interrupts passed in. No HW initialization is
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] SMP/IPI/ERAT/MB combined
On Mon, Nov 13, 2006 at 12:08:33PM -0500, Jimi Xenidis wrote: > On Nov 9, 2006, at 8:04 PM, Amos Waterland wrote: > >This patch rolls up and rebases the following patches for submission > >against current tip of tree: > > > > * Memory barrier after sp store > > * Flush the ERAT early for secondary CPUs > > * SMP and IPI support > > Good News: > On both I can run several domains, many more that I have physical > CPUs with all sorts of VIO combos. > > Bad News: > Testing on my JS21 with 4 CPUs, when I try to dump the registers for > the xen console (after a ^A*3) I get: > > ---> HANG for 3 minutes (wall clock) > (XEN) *** Dumping CPU1 state: *** > (XEN) IPI start stall: 0 ACKS to 1 SYNS > > Oddly enough, If I boot with "sync_console" there is no HANG of any > time. > > Does this work for you? > > Anyways, I cannot accept this patch if '^A*3 d' does not work, it is > too important for development. It took me a while to figure out what was going on here. The problem turned out to be that my patch that keeps dom0 Linux from resetting the IPI vectors never got merged. So C-a*3 works fine if you do it early in boot, but it stops working halfway through dom0 boot when Linux initializes IPIs to its reserved vectors and marks them disabled. I will resend the Linux patch. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Flush the ERAT early for secondary CPUs
On Fri, Nov 10, 2006 at 03:55:26AM +0100, Segher Boessenkool wrote: > > p = probability of success = .997 (897 / 900) > > q = probability of failure = .003 (1.0 - .997) > > n = number of trials = 2323 > > X = number of successes= 2323 > > > >Applying these to the binomial probability formula, we get: > > > > P(2323) = 2323! / ((2323 - 2323)! * 2323!) * .997**2323 * .003** > >(2323-2323) > > = .0009307922 > > > >So we conclude that the probability that our trials with this patch > >applied achieved exactly 2323 successes because of chance alone is . > >0009. > > Not prematurely rounding p to a useless precision gives you > > p**2323 ~ 0.000428 Sure. I just rounded early because most online tools that people would use to verify the calculation cannot deal with the higher precion. > even. And that just calculates the chance that 2323 tries all > succeed given that the chance for one to succeed is 897/900; it > doesn't compare two hypotheses at all. What? The null hypothesis is that this patch has no effect. Since the probability of observing 2323/2323 successes is .000428, we reject the null hypothesis under a 95% confidence interval. No? ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] SMP/IPI/ERAT/MB combined
This patch rolls up and rebases the following patches for submission
against current tip of tree:
* Memory barrier after sp store
* Flush the ERAT early for secondary CPUs
* SMP and IPI support
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
changeset : b30cb72ed5e2+31ae0
machines: kpblade1 cso103 cso102 kpblade7 cso98 cso99
fail: 0
transient : 1
pass: 224
total : 225
reliability : 100%
---
arch/powerpc/external.c| 32 +
arch/powerpc/mpic.c|9 -
arch/powerpc/mpic_init.c | 48
arch/powerpc/powerpc64/exceptions.S|4
arch/powerpc/setup.c | 50 +
arch/powerpc/smp.c | 135 +++--
include/asm-powerpc/mach-default/irq_vectors.h | 22
include/asm-powerpc/smp.h | 17 +++
8 files changed, 261 insertions(+), 56 deletions(-)
diff -r b30cb72ed5e2 xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Fri Nov 03 16:53:17 2006 -0500
+++ b/xen/arch/powerpc/external.c Thu Nov 09 18:02:59 2006 -0500
@@ -82,7 +82,16 @@ void do_external(struct cpu_user_regs *r
vec = xen_mpic_get_irq(regs);
-if (vec != -1) {
+if (vector_is_ipi(vec)) {
+ /* do_IRQ is fundamentally broken for reliable IPI delivery. */
+ irq_desc_t *desc = &irq_desc[vec];
+ regs->entry_vector = vec;
+ spin_lock(&desc->lock);
+ desc->handler->ack(vec);
+ desc->action->handler(vector_to_irq(vec), desc->action->dev_id, regs);
+ desc->handler->end(vec);
+ spin_unlock(&desc->lock);
+} else if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
regs->entry_vector = vec;
do_IRQ(regs);
@@ -253,3 +262,24 @@ int ioapic_guest_write(unsigned long phy
BUG_ON(val != val);
return 0;
}
+
+void send_IPI_mask(cpumask_t mask, int vector)
+{
+unsigned int cpus;
+int const bits = 8 * sizeof(cpus);
+
+switch(vector) {
+case CALL_FUNCTION_VECTOR:
+case EVENT_CHECK_VECTOR:
+ break;
+default:
+ BUG();
+ return;
+}
+
+BUG_ON(NR_CPUS > bits);
+BUG_ON(fls(mask.bits[0]) > bits);
+
+cpus = mask.bits[0];
+mpic_send_ipi(vector, cpus);
+}
diff -r b30cb72ed5e2 xen/arch/powerpc/mpic.c
--- a/xen/arch/powerpc/mpic.c Fri Nov 03 16:53:17 2006 -0500
+++ b/xen/arch/powerpc/mpic.c Thu Nov 09 18:02:59 2006 -0500
@@ -27,10 +27,6 @@
#define alloc_bootmem(x) xmalloc_bytes(x)
-#define request_irq(irq, handler, f, devname, dev_id) \
-panic("IPI requested: %d: %p: %s: %p\n", irq, handler, devname, dev_id)
-
-typedef int irqreturn_t;
#define IRQ_NONE (0)
#define IRQ_HANDLED(1)
@@ -96,11 +92,6 @@ typedef int irqreturn_t;
#endif
#include
#include
-
-static inline void smp_message_recv(int msg, struct pt_regs *regs)
-{
-return;
-}
#ifdef DEBUG
#define DBG(fmt...) printk(fmt)
diff -r b30cb72ed5e2 xen/arch/powerpc/mpic_init.c
--- a/xen/arch/powerpc/mpic_init.c Fri Nov 03 16:53:17 2006 -0500
+++ b/xen/arch/powerpc/mpic_init.c Thu Nov 09 18:02:59 2006 -0500
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "mpic_init.h"
#include "oftree.h"
#include "of-devtree.h"
@@ -358,6 +359,42 @@ static struct hw_interrupt_type *share_m
#endif
+static unsigned int mpic_startup_ipi(unsigned int irq)
+{
+mpic->hc_ipi.enable(irq);
+return 0;
+}
+
+int request_irq(unsigned int irq,
+ irqreturn_t (*handler)(int, void *, struct cpu_user_regs *),
+ unsigned long irqflags, const char * devname, void *dev_id)
+{
+int retval;
+struct irqaction *action;
+void (*func)(int, void *, struct cpu_user_regs *);
+
+action = xmalloc(struct irqaction);
+if (!action) {
+ BUG();
+ return -ENOMEM;
+}
+
+/* Xen's handler prototype is slightly different than Linux's. */
+func = (void (*)(int, void *, struct cpu_user_regs *))handler;
+
+action->handler = func;
+action->name = devname;
+action->dev_id = dev_id;
+
+retval = setup_irq(irq, action);
+if (retval) {
+ BUG();
+ xfree(action);
+}
+
+return retval;
+}
+
struct hw_interrupt_type *xen_mpic_init(struct hw_interrupt_type *xen_irq)
{
unsigned int isu_size;
@@ -397,6 +434,11 @@ struct hw_interrupt_type *xen_mpic_init(
hit = share_mpic(&mpic->hc_irq, xen_irq);
printk("%s: success\n", __func__);
+
+mpic->hc_ipi.ack = xen_irq->ack;
+mpic->hc_ipi.startup = mpic_startup_ipi;
+mpic_request_ipis();
+
return hit;
}
@@ -406,3 +448,9 @@ int xen_mpic_get_irq(struct cpu_user_reg
return mpic_get_one_irq(mpic, regs);
}
+
+int vector_is_ipi(i
[XenPPC] [PATCH] Memory barrier after sp store
Ensure that the store to pa.hype_stack_base will be seen before the store of pa to global_cpu_table[cpuid]. I have been unable to demonstrate this causing a problem in practice despite many attempts, but I think it should go in by inspection anyway. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- setup.c |1 + 1 file changed, 1 insertion(+) diff -r 9148f7816d00 xen/arch/powerpc/setup.c --- a/xen/arch/powerpc/setup.c Tue Oct 24 19:11:00 2006 -0400 +++ b/xen/arch/powerpc/setup.c Thu Nov 02 12:29:58 2006 -0500 @@ -234,6 +234,7 @@ static void init_parea(int cpuid) pa->whoami = cpuid; pa->hard_id = cpu_hard_id[cpuid]; pa->hyp_stack_base = (void *)((ulong)stack + STACK_SIZE); +mb(); /* This store has the effect of invoking secondary_cpu_init. */ global_cpu_table[cpuid] = pa; ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Flush the ERAT early for secondary CPUs
Flush the ERAT very early on secondary processors. Because Jimi has expressed skepticism about the need to do this, we provide the following empirical and statistical arguments. In the results presented below, we observe 897 passes out of 900 trials with vanilla Xen/PPC plus the SMP/IPI patch. Note that we have observed the "Synchronizing timebase hang" with pure vanilla Xen/PPC, but we test here with the SMP/IPI patch because we observe the hang to occcur more often with it. With this patch applied, we observe 2323 passes out of 2323 trials. In all cases, we use dom0 making it to a bash prompt as the success criteria. Since we have empirical evidence that the hang occurs randomly, we declare the following: p = probability of success = .997 (897 / 900) q = probability of failure = .003 (1.0 - .997) n = number of trials = 2323 X = number of successes= 2323 Applying these to the binomial probability formula, we get: P(2323) = 2323! / ((2323 - 2323)! * 2323!) * .997**2323 * .003**(2323-2323) = .0009307922 So we conclude that the probability that our trials with this patch applied achieved exactly 2323 successes because of chance alone is .0009. --- Reliability results for Xen with the SMP/IPI patch plus this ERAT flush patch applied: changeset : 25c51961bd3f+8c8eb machines: kpblade1 cso103 cso102 kpblade7 cso98 cso99 fail: 0 transient : 6 pass: 2323 total : 2329 reliability : 100% Reliability results for Xen with the SMP/IPI patch applied: changeset : 25c51961bd3f+0 machines: kpblade1 cso103 cso102 kpblade7 cso98 cso99 fail: 3 transient : 4 pass: 897 total : 904 reliability : 99% 25c51961bd3f+0/kpblade7/xen.log.2006-11-09.112408 = (XEN) Synchronizing timebase 25c51961bd3f+0/cso102/xen.log.2006-11-09.114634 === (XEN) Synchronizing timebase 25c51961bd3f+0/cso102/xen.log.2006-11-09.133343 === spinning up secondary processor #3: ping = 0x: Note that SLOF reverted to PHYP four times in the 904 runs without this patch applied, and never in the 2329 runs with this patch applied. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> exceptions.S |4 1 file changed, 4 insertions(+) diff -r 25c51961bd3f xen/arch/powerpc/powerpc64/exceptions.S --- a/xen/arch/powerpc/powerpc64/exceptions.S Thu Oct 26 20:58:55 2006 -0400 +++ b/xen/arch/powerpc/powerpc64/exceptions.S Wed Nov 08 22:10:06 2006 -0500 @@ -564,6 +564,10 @@ _GLOBAL(sleep) */ .globl spin_start spin_start: +/* We discovered by experiment that the ERAT must be flushed early. */ +isync +slbia +isync /* Do a cache flush for our text, in case the loader didn't */ LOADADDR(r9, _start) ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Timebase sync hang
I just saw a case of the timebase sync hang using vanilla Xen/PPC: changeset: 12468:9148f7816d00 tag: tip user:Jimi Xenidis <[EMAIL PROTECTED]> date:Tue Oct 24 19:11:00 2006 -0400 summary: [TOOLS][POWERPC]fixes to put back commandline in prose builder The console output looked like this: (XEN) Physical RAM map: (XEN) : 2000 (XEN) End of Xen Area: 39MiB (40608KiB) (XEN) End of RAM: 512MiB (524288KiB) (XEN) Xen Heap: 35MiB (36828KiB) (XEN) Dom Heap: 466MiB (477536KiB) (XEN) CPU[PIR:0 IPI:0 Logical:0] Hello World! (XEN) spinning up at most 16 total processors ... (XEN) Synchronizing timebase ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Timeout in the past
I have seen this on a Maple now as well: (XEN) reprogram_timer[00] Timeout in the past 0x0038879D3833 > 0x0038879CF88C (XEN) reprogram_timer[00] Timeout in the past 0x003887AE47A4 > 0x003887AD2018 ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Timeout in the past
Using tip of tree Xen/PPC I am seeing warnings like the following on JS20: (XEN) reprogram_timer[00] Timeout in the past 0x0085D8B8CC58 > 0x0085D8B845D5 ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] SMP and IPI support
On Thu, Oct 26, 2006 at 11:24:32PM -0400, Amos Waterland wrote: > Enable SMP and IPI support, including remote function invocation. It took over 1000 attempts but I did see a case of the timebase sync hang. I will try to track this down. changeset : 25c51961bd3f machines: kpblade1 cso91 cso103 pass: 1103 fail: 1 transient : 1 total : 1105 reliability : 99% 25c51961bd3f/cso103/xen.log.2006-10-27.000551 = (XEN) Min 6 (score -281), Max 7 (score 235) (XEN) Final offset: 7 (93/300) (XEN) Synchronizing timebase ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Backtace on single processor blade
We have a particular JS20 blade, named kpblade11, on which we have to flash a downlevel SLOF, and on which when Linux runs it reports only one CPU. Current tip of tree Xen/PPC appears to find the missing processor in some capacity, as the following is printed during boot. Note that this is vanilla Xen/PPC, without any SMP/IPI patches applied. boot_of_serial: ISA base: 0xf400 boot_of_serial: ISRC=0x44, but forcing poll mode [ Xen-3.0-unstable ] __ ___ ___ __ _ \ \/ /___ _ __ |___ / / _ \_ _ _ __ ___| |_ __ _| |__ | | ___ \ // _ \ '_ \|_ \| | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \ / \ __/ | | | ___) | |_| |__| |_| | | | \__ \ || (_| | |_) | | __/ /_/\_\___|_| |_| |(_)___/\__,_|_| |_|___/\__\__,_|_.__/|_|\___| CPU: 7db14ba6 DOMID: http://www.cl.cam.ac.uk/netos/xen University of Cambridge Computer Laboratory pc 4100 msr 90001000 lr ff000374 ctr srr0 srr1 Xen version 3.0-unstable ([EMAIL PROTECTED]) (gcc version 4.1.1 ()) Thu Oct 26 21:28:06 EDT 2006 r00: fd3c2000 Latest ChangeSet: Tue Oct 24 19:11:00 2006 -0400 12468:9148f7816d00 (XEN) r04: 4800 4110 (XEN) Physical RAM map: (XEN) r08: (XEN) : 2000 (XEN) r12: (XEN) End of Xen Area: 39MiB (40480KiB) (XEN) End of RAM: 512MiB (524288KiB) (XEN) r16: (XEN) r20: (XEN) r24: (XEN) r28: 0001 (XEN) Xen Heap: 35MiB (36828KiB) (XEN) dar 0x, dsisr 0x (XEN) Dom Heap: 466MiB (477664KiB) (XEN) hid4 0x (XEN) CPU[PIR:0 IPI:0 Logical:0] Hello World! (XEN) ---[ backtrace ]--- (XEN) spinning up at most 16 total processors ... (XEN) [] xen_mpic_init: start (XEN) F82D00207C3B02A6mpic: Setting up MPIC "Xen-U3-MPIC" version 1.2 at f804, max 4 CPUs (XEN) (unreliable) (XEN) mpic: ISU size: 124, shift: 7, mask: 7f (XEN) SP (7db14ba67db04aa6) is not in xen space (XEN) mpic: Initializing for 124 sources (XEN) machine_halt called (XEN) mpic: Setting up HT PICs workarounds for U3/U4 (XEN) machine_halt failed, manual powercycle required! (XEN) mpic: - HT:01.0 [0xb8] vendor 1022 device 7450 has 4 irqs (XEN) mpic: - HT:02.0 [0xb8] vendor 1022 device 7450 has 4 irqs (XEN) mpic: - HT:03.0 [0xf0] vendor 1022 device 7460 has 24 irqs (XEN) xen_mpic_init: success (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) WARNING!: Taking all secondary CPUs offline (XEN) Initializing DART 0xf8033000: tbl: 0020[0x200] entries: 0x8 (XEN) Initializing DART Model U3: reg: f8033000 word: 200200 (XEN) [00476BF0] 0044CA78 .unimplemented+0x18/0x38 (XEN) [00476C70] 0044CC80 .on_selected_cpus+0x34/0x5c (XEN) [00476CF0] 0044CBE8 .smp_call_function+0x80/0xac (XEN) [00476D80] 00426A24 .schedulers_start+0x30/0x54 (XEN) [00476E00] 0045A40C .start_of_day+0x150/0x174 (XEN) [00476E90] 0045A810 .__start_xen+0x3e0/0x6c0 (XEN) [00476F70] 0045AB84 .__start_xen_ppc+0x94/0xbc (XEN) [00477000] 0040004C _start64+0x38/0x3c (XEN) allocated RMA for Dom[0]: 0x400[0x400] ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] SMP and IPI support
Enable SMP and IPI support, including remote function invocation.
There are a number of subtle issues fixed in this patch. I believe it
is a candidate for merging.
I have tested this extensively on JS21 and model 884221X JS20 blades,
and to a degree on Maple. I would appreciate further testing on Maple
and model 884241X JS20 blades.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
changeset : 25c51961bd3f
machines: kpblade1 cso91 cso103
pass: 162
fail: 0
transient : 1
total : 163
reliability : 100%
---
arch/powerpc/external.c| 32 +
arch/powerpc/mpic.c|9 -
arch/powerpc/mpic_init.c | 48
arch/powerpc/setup.c | 49 +
arch/powerpc/smp.c | 135 +++--
include/asm-powerpc/mach-default/irq_vectors.h | 22
include/asm-powerpc/smp.h | 17 +++
7 files changed, 256 insertions(+), 56 deletions(-)
diff -r 3dfeb3e4a03f xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Fri Oct 13 11:00:32 2006 -0400
+++ b/xen/arch/powerpc/external.c Thu Oct 26 16:49:44 2006 -0400
@@ -82,7 +82,16 @@ void do_external(struct cpu_user_regs *r
vec = xen_mpic_get_irq(regs);
-if (vec != -1) {
+if (vector_is_ipi(vec)) {
+ /* do_IRQ is fundamentally broken for reliable IPI delivery. */
+ irq_desc_t *desc = &irq_desc[vec];
+ regs->entry_vector = vec;
+ spin_lock(&desc->lock);
+ desc->handler->ack(vec);
+ desc->action->handler(vector_to_irq(vec), desc->action->dev_id, regs);
+ desc->handler->end(vec);
+ spin_unlock(&desc->lock);
+} else if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
regs->entry_vector = vec;
do_IRQ(regs);
@@ -253,3 +262,24 @@ int ioapic_guest_write(unsigned long phy
BUG_ON(val != val);
return 0;
}
+
+void send_IPI_mask(cpumask_t mask, int vector)
+{
+unsigned int cpus;
+int const bits = 8 * sizeof(cpus);
+
+switch(vector) {
+case CALL_FUNCTION_VECTOR:
+case EVENT_CHECK_VECTOR:
+ break;
+default:
+ BUG();
+ return;
+}
+
+BUG_ON(NR_CPUS > bits);
+BUG_ON(fls(mask.bits[0]) > bits);
+
+cpus = mask.bits[0];
+mpic_send_ipi(vector, cpus);
+}
diff -r 3dfeb3e4a03f xen/arch/powerpc/mpic.c
--- a/xen/arch/powerpc/mpic.c Fri Oct 13 11:00:32 2006 -0400
+++ b/xen/arch/powerpc/mpic.c Mon Oct 23 22:57:51 2006 -0400
@@ -27,10 +27,6 @@
#define alloc_bootmem(x) xmalloc_bytes(x)
-#define request_irq(irq, handler, f, devname, dev_id) \
-panic("IPI requested: %d: %p: %s: %p\n", irq, handler, devname, dev_id)
-
-typedef int irqreturn_t;
#define IRQ_NONE (0)
#define IRQ_HANDLED(1)
@@ -96,11 +92,6 @@ typedef int irqreturn_t;
#endif
#include
#include
-
-static inline void smp_message_recv(int msg, struct pt_regs *regs)
-{
-return;
-}
#ifdef DEBUG
#define DBG(fmt...) printk(fmt)
diff -r 3dfeb3e4a03f xen/arch/powerpc/mpic_init.c
--- a/xen/arch/powerpc/mpic_init.c Fri Oct 13 11:00:32 2006 -0400
+++ b/xen/arch/powerpc/mpic_init.c Thu Oct 26 16:48:19 2006 -0400
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "mpic_init.h"
#include "oftree.h"
#include "of-devtree.h"
@@ -358,6 +359,42 @@ static struct hw_interrupt_type *share_m
#endif
+static unsigned int mpic_startup_ipi(unsigned int irq)
+{
+mpic->hc_ipi.enable(irq);
+return 0;
+}
+
+int request_irq(unsigned int irq,
+ irqreturn_t (*handler)(int, void *, struct cpu_user_regs *),
+ unsigned long irqflags, const char * devname, void *dev_id)
+{
+int retval;
+struct irqaction *action;
+void (*func)(int, void *, struct cpu_user_regs *);
+
+action = xmalloc(struct irqaction);
+if (!action) {
+ BUG();
+ return -ENOMEM;
+}
+
+/* Xen's handler prototype is slightly different than Linux's. */
+func = (void (*)(int, void *, struct cpu_user_regs *))handler;
+
+action->handler = func;
+action->name = devname;
+action->dev_id = dev_id;
+
+retval = setup_irq(irq, action);
+if (retval) {
+ BUG();
+ xfree(action);
+}
+
+return retval;
+}
+
struct hw_interrupt_type *xen_mpic_init(struct hw_interrupt_type *xen_irq)
{
unsigned int isu_size;
@@ -397,6 +434,11 @@ struct hw_interrupt_type *xen_mpic_init(
hit = share_mpic(&mpic->hc_irq, xen_irq);
printk("%s: success\n", __func__);
+
+mpic->hc_ipi.ack = xen_irq->ack;
+mpic->hc_ipi.startup = mpic_startup_ipi;
+mpic_request_ipis();
+
return hit;
}
@@ -406,3 +448,9 @@ int xen_mpic_get_irq(struct cpu_user_reg
return mp
[XenPPC] [PATCH] Remove timestamp from xen_maple_defconfig
Remove timestamp from xen_maple_defconfig to reduce merge conflicts in client trees. Generated with: KCONFIG_NOTIMESTAMP=1 make xen_maple_defconfig Thanks to Segher Boessenkool for the suggestion. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- xen_maple_defconfig |1 - 1 file changed, 1 deletion(-) diff -r f4d382795e57 arch/powerpc/configs/xen_maple_defconfig --- a/arch/powerpc/configs/xen_maple_defconfig Wed Oct 25 17:22:54 2006 -0400 +++ b/arch/powerpc/configs/xen_maple_defconfig Thu Oct 26 12:25:27 2006 -0400 @@ -1,7 +1,6 @@ # # Automatically generated make config: don't edit # Linux kernel version: 2.6.17 -# Thu Oct 12 16:16:49 2006 # CONFIG_PPC64=y CONFIG_64BIT=y ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [linux-ppc-2.6] [LINUX][XEN][POWEPRC] def config changes
On Wed, Oct 25, 2006 at 09:02:29PM -0400, Maria Butrico wrote: > I thought we diligently updated this file by hand. We edit it all the > time. The comment is flat out wrong. The file is in fact automatically generated. Nobody should ever edit a defconfig by hand, since the Kconfig logic that turns on or off certain options in response to the change would then have to be run by every end user, resulting in a .config different from the defconfig in every developer tree. If we edit a config by hand, we edit .config in the root, then run make, then copy .config to the desired defconfig. > Why did you change the date? Jimi did not change the date by hand. The Kconfig logic discussed above did so, and I believe that the myriad benefits of that logic outweigh the cost of resolving the trivial merge conflict caused by the date. Perhaps we can investigate removing the date insertion logic if this becomes a real maintainer burden. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Enable SMP and IPIs
On Tue, Oct 24, 2006 at 05:54:47AM -0400, Jimi Xenidis wrote: > On Oct 24, 2006, at 12:22 AM, Amos Waterland wrote: > > Note that the flurry of IPIs generated by domain creation seems to > > be a waste of time. Xen on x86 doesn't actually to do anything in > > response to them, so I have made Xen on PPC properly deliver the > > event check but then do nothing as well. Comments? What do you think we should be doing in response to these event check IPIs? I thought that there were hopes of improving interactivity of single-cpu systems ... > I know this is RFC, and you know I'm a bug fan of BUG(), but > eventually I'd expect: > BUG_ON(!action) > if (!action) > return -ENOMEM I think your comments here are predicated on the assumption that BUG macros become noops when not compiling with debug. That is not the case in Xen, so I thought it was a waste of time to put in extra code. Do you think it might someday not be the case? ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Use xvc to boot on bare metal
On Tue, Oct 24, 2006 at 03:18:26PM -0500, Hollis Blanchard wrote: > I don't see the dependency? It depends on that patch in that it is irrelevent without that patch. Linux would not make it far enough to worry about console. > This doesn't seem to be PowerPC-specific. Why don't you send this to > xen-devel? You are correct: it is not PowerPC-specific, and I have posted it to xen-devel. We are currently trying to get LANANA to assign the minor number, but Redhat is using the patch with a minor in the experimental range. I only posted it here because some people who use Xen/PPC needed it. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Use xvc to boot on bare metal
This patch allows Linux compiled with xen_maple_defconfig to boot on
both bare hardware and as a dom0. I had to respin it to keep up with changes.
Note that this depends on a xen_start_info patch soon to be posted by Jimi.
To use, put `xencons=xvc0' in your Linux bootargs, and run
`mknod /dev/xvc0 c 204 187' in your rootfs. Built and booted up to
nfsroot on a JS21 ppc64 blade both on bare hardware and as a dom0.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/configs/xen_maple_defconfig | 12 ++--
drivers/xen/console/console.c| 27 ++-
include/linux/major.h|1 +
3 files changed, 37 insertions(+), 3 deletions(-)
diff -r 968ced1469e8 drivers/xen/console/console.c
--- a/drivers/xen/console/console.c Tue Oct 17 17:03:31 2006 -0400
+++ b/drivers/xen/console/console.c Mon Oct 23 22:12:03 2006 -0400
@@ -63,13 +63,20 @@
* 'xencons=off' [XC_OFF]: Console is disabled.
* 'xencons=tty' [XC_TTY]: Console attached to '/dev/tty[0-9]+'.
* 'xencons=ttyS' [XC_SERIAL]: Console attached to '/dev/ttyS[0-9]+'.
+ * 'xencons=xvc' [XC_XVC]: Console attached to '/dev/xvc[0-9]+'.
* [XC_DEFAULT]: DOM0 -> XC_SERIAL ; all others -> XC_TTY.
*
* NB. In mode XC_TTY, we create dummy consoles for tty2-63. This suppresses
* warnings from standard distro startup scripts.
*/
-static enum { XC_OFF, XC_DEFAULT, XC_TTY, XC_SERIAL } xc_mode = XC_DEFAULT;
+static enum { XC_OFF, XC_DEFAULT, XC_TTY, XC_SERIAL, XC_XVC }
+xc_mode = XC_DEFAULT;
static int xc_num = -1;
+
+/* If we are in XC_XVC mode (a virtual console at /dev/xvcX), we need to
+ * comply with Lanana and use a minor under the low density serial major.
+ */
+#define XEN_XVC_MINOR 187
#ifdef CONFIG_MAGIC_SYSRQ
static unsigned long sysrq_requested;
@@ -85,6 +92,8 @@ static int __init xencons_setup(char *st
xc_mode = XC_SERIAL;
else if (!strncmp(str, "tty", 3))
xc_mode = XC_TTY;
+ else if (!strncmp(str, "xvc", 3))
+ xc_mode = XC_XVC;
else if (!strncmp(str, "off", 3))
xc_mode = XC_OFF;
@@ -95,6 +104,11 @@ static int __init xencons_setup(char *st
xc_num = n;
break;
case XC_TTY:
+ n = simple_strtol(str+3, &q, 10);
+ if (q > (str + 3))
+ xc_num = n;
+ break;
+ case XC_XVC:
n = simple_strtol(str+3, &q, 10);
if (q > (str + 3))
xc_num = n;
@@ -199,6 +213,12 @@ static int __init xen_console_init(void)
}
switch (xc_mode) {
+ case XC_XVC:
+ strcpy(kcons_info.name, "xvc");
+ if (xc_num == -1)
+ xc_num = 0;
+ break;
+
case XC_SERIAL:
strcpy(kcons_info.name, "ttyS");
if (xc_num == -1)
@@ -595,6 +615,11 @@ static int __init xencons_init(void)
DRV(xencons_driver)->name= "ttyS";
DRV(xencons_driver)->minor_start = 64 + xc_num;
DRV(xencons_driver)->name_base = 0 + xc_num;
+ } else if (xc_mode == XC_XVC) {
+ DRV(xencons_driver)->name= "xvc";
+ DRV(xencons_driver)->major = LOW_DENS_SERIAL_MAJOR;
+ DRV(xencons_driver)->minor_start = XEN_XVC_MINOR;
+ DRV(xencons_driver)->name_base = xc_num;
} else {
DRV(xencons_driver)->name= "tty";
DRV(xencons_driver)->minor_start = xc_num;
diff -r 968ced1469e8 include/linux/major.h
--- a/include/linux/major.h Tue Oct 17 17:03:31 2006 -0400
+++ b/include/linux/major.h Mon Oct 23 22:12:03 2006 -0400
@@ -158,6 +158,7 @@
#define MSR_MAJOR 202
#define CPUID_MAJOR203
+#define LOW_DENS_SERIAL_MAJOR 204
#define OSST_MAJOR 206 /* OnStream-SCx0 SCSI tape */
diff -r 968ced1469e8 arch/powerpc/configs/xen_maple_defconfig
--- a/arch/powerpc/configs/xen_maple_defconfig Tue Oct 17 17:03:31 2006 -0400
+++ b/arch/powerpc/configs/xen_maple_defconfig Tue Oct 24 10:55:38 2006 -0400
@@ -1,7 +1,7 @@
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.17
-# Sat Oct 7 08:37:53 2006
+# Tue Oct 24 10:51:54 2006
#
CONFIG_PPC64=y
CONFIG_64BIT=y
@@ -711,10 +711,18 @@ CONFIG_HW_CONSOLE=y
#
# Serial drivers
#
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_8250_PCI=y
+CONFIG_SERIAL_8250_NR_UARTS=4
+CONFIG_SERIAL_8250_RUNTIME_UARTS=4
+# CONFIG_SERIAL_8250_EXTENDED is not set
#
# Non-8250 serial port support
#
+CONFIG_SERIAL_CORE=y
+CONFIG_S
[XenPPC] [PATCH] Enable SMP and IPIs
This patch enables SMP and IPIs. It works reliably on JS20 and JS21
blades. It is not quite ready for submission, but I would greatly
appreciate any comments.
Note that the flurry of IPIs generated by domain creation seems to be a
waste of time. Xen on x86 doesn't actually to do anything in response
to them, so I have made Xen on PPC properly deliver the event check but
then do nothing as well. Comments?
The important thing this patch is missing is the ability to invoke
functions on a remote CPU, and I have left it out because I am not yet
happy with the convention, which is to grab a lock, put the target
address in a single global shared variable, invoke a memory barrier, and
send the IPI. I am hoping to avoid a lock around that operation by
making a per-IPI-destination-CPU structure with lock, address, and ack
fields. Comments?
---
changeset : cc0cd5518e10
machines: cso103 kpblade1 cso91
pass: 35
fail: 0
transient : 0
total : 35
reliability : 100.0%
---
arch/powerpc/mpic.c|9 ---
arch/powerpc/mpic_init.c | 42 +
arch/powerpc/setup.c | 40
arch/powerpc/smp.c | 59 -
include/asm-powerpc/mach-default/irq_vectors.h | 11 ++--
include/asm-powerpc/smp.h | 10
6 files changed, 136 insertions(+), 35 deletions(-)
diff -r 3dfeb3e4a03f xen/arch/powerpc/mpic.c
--- a/xen/arch/powerpc/mpic.c Fri Oct 13 11:00:32 2006 -0400
+++ b/xen/arch/powerpc/mpic.c Mon Oct 23 22:57:51 2006 -0400
@@ -27,10 +27,6 @@
#define alloc_bootmem(x) xmalloc_bytes(x)
-#define request_irq(irq, handler, f, devname, dev_id) \
-panic("IPI requested: %d: %p: %s: %p\n", irq, handler, devname, dev_id)
-
-typedef int irqreturn_t;
#define IRQ_NONE (0)
#define IRQ_HANDLED(1)
@@ -96,11 +92,6 @@ typedef int irqreturn_t;
#endif
#include
#include
-
-static inline void smp_message_recv(int msg, struct pt_regs *regs)
-{
-return;
-}
#ifdef DEBUG
#define DBG(fmt...) printk(fmt)
diff -r 3dfeb3e4a03f xen/arch/powerpc/mpic_init.c
--- a/xen/arch/powerpc/mpic_init.c Fri Oct 13 11:00:32 2006 -0400
+++ b/xen/arch/powerpc/mpic_init.c Mon Oct 23 15:32:55 2006 -0400
@@ -358,6 +358,43 @@ static struct hw_interrupt_type *share_m
#endif
+static inline struct mpic * mpic_from_ipi(unsigned int ipi)
+{
+ return container_of(irq_desc[ipi].handler, struct mpic, hc_ipi);
+}
+
+static unsigned int mpic_startup_ipi(unsigned int irq)
+{
+printk("%s\n", __func__);
+struct mpic *pic = mpic_from_ipi(irq);
+pic->hc_ipi.enable(irq);
+return 0;
+}
+
+int request_irq(unsigned int irq,
+ irqreturn_t (*handler)(int, void *, struct cpu_user_regs *),
+ unsigned long irqflags, const char * devname, void *dev_id)
+{
+int retval;
+struct irqaction *action;
+
+printk("%s: %d: %p: %s: %p\n", __func__, irq, handler, devname, dev_id);
+
+action = xmalloc_bytes(sizeof(struct irqaction));
+if (!action)
+ BUG();
+
+action->handler = (void (*)(int, void *, struct cpu_user_regs *))handler;
+action->name = devname;
+action->dev_id = dev_id;
+
+retval = setup_irq(irq, action);
+if (retval)
+ BUG();
+
+return retval;
+}
+
struct hw_interrupt_type *xen_mpic_init(struct hw_interrupt_type *xen_irq)
{
unsigned int isu_size;
@@ -397,6 +434,11 @@ struct hw_interrupt_type *xen_mpic_init(
hit = share_mpic(&mpic->hc_irq, xen_irq);
printk("%s: success\n", __func__);
+
+mpic->hc_ipi.ack = xen_irq->ack;
+mpic->hc_ipi.startup = mpic_startup_ipi;
+mpic_request_ipis();
+
return hit;
}
diff -r 3dfeb3e4a03f xen/arch/powerpc/setup.c
--- a/xen/arch/powerpc/setup.c Fri Oct 13 11:00:32 2006 -0400
+++ b/xen/arch/powerpc/setup.c Fri Oct 13 19:00:34 2006 -0400
@@ -37,6 +37,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -88,6 +89,8 @@ struct ns16550_defaults ns16550;
extern char __per_cpu_start[], __per_cpu_data_end[], __per_cpu_end[];
+static struct domain *idle_domain;
+
volatile struct processor_area * volatile global_cpu_table[NR_CPUS];
int is_kernel_text(unsigned long addr)
@@ -159,8 +162,6 @@ static void percpu_free_unused_areas(voi
static void __init start_of_day(void)
{
-struct domain *idle_domain;
-
init_IRQ();
scheduler_init();
@@ -174,23 +175,6 @@ static void __init start_of_day(void)
/* for some reason we need to set our own bit in the thread map */
cpu_set(0, cpu_sibling_map[0]);
-
-percpu_free_unused_areas();
-
-{
-/* FIXME: Xen assumes that an online CPU is a schedualable
- * CPU, but we just are not there yet. Remove this fragment when
- * scheduling processors actually works. */
-int cpuid;
-
-printk("WARNING!: Taking all secondary CPUs offline\n
[XenPPC] xen_maple_defconfig fails to boot on bare metal
Compiling the current code from linux-ppc-2.6.hg with xen_maple_defconfig and booting on both JS20 and JS21 produces this: PID hash table entries: 4096 (order: 12, 32768 bytes) Maple: Found RTC at IO 0x1070 cpu 0x0: Vector: 300 (Data Access) at [c0002ffe3b60] pc: c0515128: .loopback_init+0x228/0x280 lr: c0009348: .init+0x1d8/0x430 sp: c0002ffe3de0 msr: 90009032 dar: 30 dsisr: 4000 current = 0xc00277fb47f0 paca= 0xc0538100 pid = 1, comm = swapper The tip is: changeset: 33287:968ced1469e8 tag: tip user:Jimi Xenidis <[EMAIL PROTECTED]> date:Tue Oct 17 17:03:31 2006 -0400 summary: [LINUX][XEN][POWERPC] remove mpic pointer hack. I'll debug this soon unless somebody beats me to it. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] dom0 memory size results
On a JS21 with 8GB of RAM, we attempt 8 runs with the following dom0 memory requests: 128M 192M 256M 512M 700M 1G 2G 7G. I'm surprised we need as much as 256M just to load dom0, but happy that 2G works now. --- changeset : d18a0c0b77d7 machines: cso104 pass: 5 fail: 3 transient : 0 total : 8 reliability : 62.0% d18a0c0b77d7/cso104/xen.log.2006-10-18.171142 = bootargs = xen dom0_mem=128M -- ip=9.2.78.104::9.2.78.1:255.255.255.0:cso104:eth1:off ro root=/dev/nfs nfsroot=9.2.208.86:/home/apw/devel/nfsroots/sles10-rootfs-2006-10-18 init=/sbin/quickinit Out of Memory: Kill process 2 (migration/0) score 0 and children. Out of Memory: Kill process 2 (migration/0) score 0 and children. Out of Memory: Kill process 2 (mi d18a0c0b77d7/cso104/xen.log.2006-10-18.171459 = bootargs = xen dom0_mem=192M -- ip=9.2.78.104::9.2.78.1:255.255.255.0:cso104:eth1:off ro root=/dev/nfs nfsroot=9.2.208.86:/home/apw/devel/nfsroots/sles10-rootfs-2006-10-18 init=/sbin/quickinit Out of Memory: Kill process 2 (migration/0) score 0 and children. Out of Memory: Kill process 2 (migration/0) score 0 and children. Out of d18a0c0b77d7/cso104/xen.log.2006-10-18.173445 = bootargs = xen dom0_mem=7G -- ip=9.2.78.104::9.2.78.1:255.255.255.0:cso104:eth1:off ro root=/dev/nfs nfsroot=9.2.208.86:/home/apw/devel/nfsroots/sles10-rootfs-2006-10-18 init=/sbin/quickinit Device tree struct 0x01b2a000 -> 0x01b2c000 Calling quiesce ... returning from prom_init ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Automated reliability report for Xen/PPC
An automated process has booted Xen on three JS21 blades a total of 663 times, recording 0 failures and 640 passes, using a correctness criteria of the creation of four domU's. changeset : 3dfeb3e4a03f machines: cso96 cso97 cso102 cso103 cso104 pass: 640 fail: 0 transient : 23 total : 663 reliability : 100.0% The version of Xen used was: changeset: 12407:3dfeb3e4a03f tag: tip parent: 12406:9372353e3b09 parent: 12405:ff3ab3fc8024 user:Jimi Xenidis <[EMAIL PROTECTED]> date:Fri Oct 13 11:00:32 2006 -0400 summary: merge Note that cso96 and cso97 were renamed to cso102 and cso103 in the middle of the run, so only three physical blades were tested. Note that in previous reports the mpic spurious interrupt panic was classified as a transient failure. It this report it has no such exemption, and we observed zero occurrences. For reference, we include below the xen.rly configuration file. --- # This is a rly configuration file for automated Xen reliability testing ## # If we make it to a shell prompt we consider it a pass PASS=".+:/ #" ## # Ignore the timestamp printed by script IGNORE="Script done on" # Ignore empty lines in the log IGNORE="$IGNORE|^$" # Ignore the control character used to make pretty green output IGNORE="$IGNORE|80C10D1" # Ignore hwconsole explaining that it has timed out IGNORE="$IGNORE|hwconsole: stopping execution because of timeout" ## # A slow NFS server can cause us to be slow during init TRANSIENT="0 0" # Another symptom of a slow NFS server TRANSIENT="$TRANSIENT| xm vcpu-list" # Yet another symptom of a slow NFS server TRANSIENT="$TRANSIENT|^\+$" # Yet another symptom of a slow NFS server TRANSIENT="$TRANSIENT|Started domain " # We reverted to the primary firmware bank TRANSIENT="$TRANSIENT|C400E2" # The SLOF tftp code seems to get itself into a slow mode sometimes TRANSIENT="$TRANSIENT|Receiving data: " ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Skip reset of IPI vectors
Rather than try to make a xen_mpic_init and duplicate all the associated
machinery it pulls in, this patch introduces a feature flag. I tried
the first approach, and it results in a lot of code duplication, which
is a worse maintenence burden than this, I believe.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
arch/powerpc/platforms/xen/evtchn.c |1 +
arch/powerpc/sysdev/mpic.c |2 +-
include/asm-powerpc/mpic.h |2 ++
3 files changed, 4 insertions(+), 1 deletion(-)
diff -r f0be2cc05103 arch/powerpc/platforms/xen/evtchn.c
--- a/arch/powerpc/platforms/xen/evtchn.c Fri Oct 13 12:36:39 2006 -0400
+++ b/arch/powerpc/platforms/xen/evtchn.c Fri Oct 13 18:39:46 2006 -0400
@@ -106,6 +106,7 @@ static __init void xen_share_maple_init_
mpic = mpic_alloc(opic_addr,
MPIC_PRIMARY | MPIC_BIG_ENDIAN |
+ MPIC_SKIP_IPI_INIT |
MPIC_BROKEN_U3 /* | MPIC_WANTS_RESET */,
0, 0, 128, 128, senses, 128, "Xen-U3-MPIC");
BUG_ON(mpic == NULL);
diff -r f0be2cc05103 arch/powerpc/sysdev/mpic.c
--- a/arch/powerpc/sysdev/mpic.cFri Oct 13 12:36:39 2006 -0400
+++ b/arch/powerpc/sysdev/mpic.cFri Oct 13 18:38:07 2006 -0400
@@ -743,7 +743,7 @@ void __init mpic_init(struct mpic *mpic)
/* Initialize IPIs to our reserved vectors and mark them disabled for
now */
mpic_test_broken_ipi(mpic);
- for (i = 0; i < 4; i++) {
+ for (i = 0; i < 4 && !(mpic->flags & MPIC_SKIP_IPI_INIT); i++) {
mpic_ipi_write(i,
MPIC_VECPRI_MASK |
(10 << MPIC_VECPRI_PRIORITY_SHIFT) |
diff -r f0be2cc05103 include/asm-powerpc/mpic.h
--- a/include/asm-powerpc/mpic.hFri Oct 13 12:36:39 2006 -0400
+++ b/include/asm-powerpc/mpic.hFri Oct 13 18:38:13 2006 -0400
@@ -190,6 +190,8 @@ struct mpic
#define MPIC_BROKEN_IPI0x0008
/* MPIC wants a reset */
#define MPIC_WANTS_RESET 0x0010
+/* Skip reset of IPI vectors during init */
+#define MPIC_SKIP_IPI_INIT 0x0020
/* Allocate the controller structure and setup the linux irq descs
* for the range if interrupts passed in. No HW initialization is
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix Maple SMP spurious interrupts assert
This lets Xen with the SMP patch boot reliably on the Maple-D platform I
have access to.
I believe that the long-term strategy for Xen/PPC is to completely
virtualize the PIC, but until that happens, this patch will I believe
allow the SMP patch to go in.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
Makefile |4
mpic.c |2 ++
2 files changed, 6 insertions(+)
diff -r b9e38b262f64 arch/powerpc/sysdev/Makefile
--- a/arch/powerpc/sysdev/Makefile Sun Oct 08 14:22:12 2006 -0400
+++ b/arch/powerpc/sysdev/Makefile Wed Oct 11 19:35:53 2006 -0400
@@ -1,5 +1,9 @@ ifeq ($(CONFIG_PPC64),y)
ifeq ($(CONFIG_PPC64),y)
EXTRA_CFLAGS += -mno-minimal-toc
+endif
+
+ifeq ($(CONFIG_PPC_XEN),y)
+EXTRA_CFLAGS += -DCONFIG_PPC_XEN
endif
obj-$(CONFIG_MPIC) += mpic.o
diff -r b9e38b262f64 arch/powerpc/sysdev/mpic.c
--- a/arch/powerpc/sysdev/mpic.cSun Oct 08 14:22:12 2006 -0400
+++ b/arch/powerpc/sysdev/mpic.cWed Oct 11 19:40:07 2006 -0400
@@ -619,6 +619,7 @@ struct mpic * __init mpic_alloc(unsigned
mpic->tmregs = mpic->gregs + ((MPIC_TIMER_BASE - MPIC_GREG_BASE) >> 2);
BUG_ON(mpic->gregs == NULL);
+#ifndef CONFIG_PPC_XEN
/* Reset */
if (flags & MPIC_WANTS_RESET) {
mpic_write(mpic->gregs, MPIC_GREG_GLOBAL_CONF_0,
@@ -628,6 +629,7 @@ struct mpic * __init mpic_alloc(unsigned
& MPIC_GREG_GCONF_RESET)
mb();
}
+#endif
/* Read feature register, calculate num CPUs and, for non-ISU
* MPICs, num sources as well. On ISU MPICs, sources are counted
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: Status summary Was: [XenPPC] Cannot boot from local disk
On Sat, Oct 07, 2006 at 01:56:56AM +0900, Kiyokuni Kawachiya wrote:
> > 2. The SMP patch apparently causes dom0 Linux to fail
> > on Kawachiya's JS20, but no other JS20.
>
> > is not solved yet, may depend on my JS20. sent the boot log
Hi Kawachiya. If you get a chance, could you please apply this patch to
the Linux tree you use for dom0 and boot without the "nosmp" option?
diff -r b9e38b262f64 arch/powerpc/sysdev/mpic.c
--- a/arch/powerpc/sysdev/mpic.cSun Oct 08 14:22:12 2006 -0400
+++ b/arch/powerpc/sysdev/mpic.cWed Oct 11 13:22:07 2006 -0400
@@ -619,6 +619,7 @@ struct mpic * __init mpic_alloc(unsigned
mpic->tmregs = mpic->gregs + ((MPIC_TIMER_BASE - MPIC_GREG_BASE) >> 2);
BUG_ON(mpic->gregs == NULL);
+#if 0
/* Reset */
if (flags & MPIC_WANTS_RESET) {
mpic_write(mpic->gregs, MPIC_GREG_GLOBAL_CONF_0,
@@ -628,6 +629,7 @@ struct mpic * __init mpic_alloc(unsigned
& MPIC_GREG_GCONF_RESET)
mb();
}
+#endif
/* Read feature register, calculate num CPUs and, for non-ISU
* MPICs, num sources as well. On ISU MPICs, sources are counted
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] mpic reset
I don't see a definitive statement in the mpic spec. Is it ok to reset
the mpic an arbitrary number of times, or even more than once, after
power on? We are allowing dom0 to reset it at the moment ...
> diff -r b9e38b262f64 arch/powerpc/sysdev/mpic.c
> --- a/arch/powerpc/sysdev/mpic.c Sun Oct 08 14:22:12 2006 -0400
> +++ b/arch/powerpc/sysdev/mpic.c Tue Oct 10 18:10:53 2006 -0400
> @@ -619,6 +619,7 @@ struct mpic * __init mpic_alloc(unsigned
> mpic->tmregs = mpic->gregs + ((MPIC_TIMER_BASE - MPIC_GREG_BASE) >> 2);
> BUG_ON(mpic->gregs == NULL);
>
> +#if 0
> /* Reset */
> if (flags & MPIC_WANTS_RESET) {
> mpic_write(mpic->gregs, MPIC_GREG_GLOBAL_CONF_0,
> @@ -628,6 +629,7 @@ struct mpic * __init mpic_alloc(unsigned
> & MPIC_GREG_GCONF_RESET)
> mb();
> }
> +#endif
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH/RFC] Initial IPI framework
This is the minimal set of changes/hacks necessary to get interprocessor
interrupts delivered reliably.
This is certainly not for submission, I am just posting it because I
would appreciate any feedback on the direction this is headed.
Basically, we are "sharing" the mpic less, but I have not ripped it
entirely out of dom0 and made a proper virtual PIC.
---
mpic.c |4
1 file changed, 4 insertions(+)
diff -r b9e38b262f64 arch/powerpc/sysdev/mpic.c
--- a/arch/powerpc/sysdev/mpic.cSun Oct 08 14:22:12 2006 -0400
+++ b/arch/powerpc/sysdev/mpic.cTue Oct 10 18:10:53 2006 -0400
@@ -619,6 +619,7 @@ struct mpic * __init mpic_alloc(unsigned
mpic->tmregs = mpic->gregs + ((MPIC_TIMER_BASE - MPIC_GREG_BASE) >> 2);
BUG_ON(mpic->gregs == NULL);
+#if 0
/* Reset */
if (flags & MPIC_WANTS_RESET) {
mpic_write(mpic->gregs, MPIC_GREG_GLOBAL_CONF_0,
@@ -628,6 +629,7 @@ struct mpic * __init mpic_alloc(unsigned
& MPIC_GREG_GCONF_RESET)
mb();
}
+#endif
/* Read feature register, calculate num CPUs and, for non-ISU
* MPICs, num sources as well. On ISU MPICs, sources are counted
@@ -741,6 +743,7 @@ void __init mpic_init(struct mpic *mpic)
(MPIC_VEC_TIMER_0 + i));
}
+#if 0
/* Initialize IPIs to our reserved vectors and mark them disabled for
now */
mpic_test_broken_ipi(mpic);
for (i = 0; i < 4; i++) {
@@ -755,6 +758,7 @@ void __init mpic_init(struct mpic *mpic)
irq_desc[mpic->ipi_offset+i].chip = &mpic->hc_ipi;
#endif /* CONFIG_SMP */
}
+#endif
/* Initialize interrupt sources */
if (mpic->irq_count == 0)
---
exceptions.c |3 +++
external.c |8 +++-
mpic.c | 24 +---
mpic_init.c |5 -
setup.c | 50 ++
smp.c|7 +--
6 files changed, 70 insertions(+), 27 deletions(-)
diff -r 3583d2701405 xen/arch/powerpc/exceptions.c
--- a/xen/arch/powerpc/exceptions.c Sun Oct 08 11:52:28 2006 -0400
+++ b/xen/arch/powerpc/exceptions.c Tue Oct 10 18:11:10 2006 -0400
@@ -26,6 +26,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -44,6 +45,8 @@ void do_timer(struct cpu_user_regs *regs
/* Set HDEC high so it stops firing and can be reprogrammed by
* set_preempt() */
mthdec(INT_MAX);
+if (mftb() % 2048 == 0)
+mpic_send_ipi(3,2);
raise_softirq(TIMER_SOFTIRQ);
}
diff -r 3583d2701405 xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Sun Oct 08 11:52:28 2006 -0400
+++ b/xen/arch/powerpc/external.c Tue Oct 10 11:27:30 2006 -0400
@@ -85,7 +85,13 @@ void do_external(struct cpu_user_regs *r
if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
regs->entry_vector = vec;
-do_IRQ(regs);
+if (vec != 131)
+ do_IRQ(regs);
+else {
+ printk("%s: got IPI: cpu #%d!\n", __func__, smp_processor_id());
+ void mpic_end_ipi(unsigned int irq);
+ mpic_end_ipi(vec);
+}
BUG_ON(mfmsr() & MSR_EE);
spur_count = 0;
diff -r 3583d2701405 xen/arch/powerpc/mpic.c
--- a/xen/arch/powerpc/mpic.c Sun Oct 08 11:52:28 2006 -0400
+++ b/xen/arch/powerpc/mpic.c Tue Oct 10 11:30:04 2006 -0400
@@ -27,8 +27,6 @@
#define alloc_bootmem(x) xmalloc_bytes(x)
-#define request_irq(irq, handler, f, devname, dev_id) \
-panic("IPI requested: %d: %p: %s: %p\n", irq, handler, devname, dev_id)
typedef int irqreturn_t;
@@ -598,7 +596,8 @@ static void mpic_disable_ipi(unsigned in
/* NEVER disable an IPI... that's just plain wrong! */
}
-static void mpic_end_ipi(unsigned int irq)
+void mpic_end_ipi(unsigned int irq);
+void mpic_end_ipi(unsigned int irq)
{
struct mpic *mpic = mpic_from_ipi(irq);
@@ -1065,6 +1064,25 @@ int mpic_get_irq(struct pt_regs *regs)
#ifdef CONFIG_SMP
+
+#define SA_INTERRUPT 0x2000u
+
+/* request_irq - allocate an interrupt line
+ * @irq: Interrupt line to allocate
+ * @handler: Function to be called when the IRQ occurs
+ * @irqflags: Interrupt type flags
+ * @devname: An ascii name for the claiming device
+ * @dev_id: A cookie passed back to the handler function
+ */
+static int request_irq(unsigned int irq,
+irqreturn_t (*handler)(int, void *, struct pt_regs *),
+unsigned long irqflags, const char * devname, void *dev_id)
+{
+printk("IPI requested: %d: %p: %s: %p\n", irq, handler, devname, dev_id);
+mpic_enable_ipi(irq);
+return 0;
+}
+
void mpic_request_ipis(void)
{
struct mpic *mpic = mpic_primary;
diff -r 3583d2701405 xen/arch/powerpc/mpic_init.c
--- a/xen/arch/powerpc/mpic_init.c Sun Oct 08 11:52:28 2006 -0400
+++ b/xen/arch/powerpc/mpic_init.c Mon Oct 0
[XenPPC] NAP state
Just as a sanity check, is the intention is to put the processor in the
NAP state when .sleep() is invoked? If so, is the following behavior
expected?
@@ -395,12 +396,34 @@ static void __init __start_xen(multiboot
console_endboot();
+while (1) {
+long before, after;
+before = mftb(); sleep(); after = mftb();
+printk("hid0 = %lx: slept for %ld timebase ticks (%ld seconds)\n",
+ mfhid0(), after - before, (after - before) / timebase_freq);
+}
/* Hide UART from DOM0 if we're using it */
serial_endboot();
(XEN) hid0 = 5101018000: slept for 1952282840 timebase ticks (9 seconds)
(XEN) hid0 = 5101018000: slept for 52834590 timebase ticks (0 seconds)
(XEN) hid0 = 5101018000: slept for 2078791813 timebase ticks (10 seconds)
(XEN) hid0 = 5101018000: slept for 52797943 timebase ticks (0 seconds)
(XEN) hid0 = 5101018000: slept for 2078719926 timebase ticks (10 seconds)
(XEN) hid0 = 5101018000: slept for 52860659 timebase ticks (0 seconds)
(XEN) hid0 = 5101018000: slept for 2078655488 timebase ticks (10 seconds)
(XEN) hid0 = 5101018000: slept for 52932165 timebase ticks (0 seconds)
(XEN) hid0 = 5101018000: slept for 2078598776 timebase ticks (10 seconds)
...
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] Spurious interrupt count
I am having some doubts about this code in external.c:
vec = xen_mpic_get_irq(regs);
if (vec != -1) {
...
spur_count = 0;
} else {
++spur_count;
if (spur_count > 100)
panic("Too many (%d) spurrious interrupts in a row\n");
}
When I do this:
--- a/xen/arch/powerpc/external.c Fri Oct 06 12:15:06 2006 -0500
+++ b/xen/arch/powerpc/external.c Sat Oct 07 00:35:01 2006 -0400
@@ -84,6 +84,7 @@ void do_external(struct cpu_user_regs *r
if (vec != -1) {
DBG("EE:0x%lx isrc: %d\n", regs->msr, vec);
+BUG();
regs->entry_vector = vec;
do_IRQ(regs);
I make it all the way to here in dom0 Linux before the BUG asserts:
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
AMD8111: IDE controller at PCI slot :00:04.1
AMD8111: chipset revision 3
AMD8111: :00:04.1 (rev 03) UDMA133 controller
AMD8111: 100% native mode on irq 16
ide0: BM-DMA at 0x8020-0x8027, BIOS settings: hda:pio, hdb:pio
ide1: BM-DMA at 0x8028-0x802f, BIOS settings: hdc:pio, hdd:pio
hda: FUJITSU MHT2040AS, ATA DISK drive
ide0 at 0x8000-0x8007,0x800a on irq 16
hda: max request size: 128KiB
(XEN) BUG at external.c:87
The code of concern seems well suited to catch a spurious interrupt
storm, since when a valid external interrupt is recieved, the counter is
reset. But given that we don't take our first external until way into
dom0's boot, we are actually asserting that 100 spurious interrupts
won't be received over a fairly long period of time.
I can't find any documents regarding expected spurious interrupt rates.
Can anyone with knowledge in this area comment about the code of concern?
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: SMP patch? Was: [XenPPC] Cannot boot from local disk
On Sat, Oct 07, 2006 at 01:36:18AM +0900, Kiyokuni Kawachiya wrote: > > > Could you synchronize the console here, there are two ways to do this: > > > 1) add "sync_console" to the cmdline _before_ "--" > > > 2) apply the follow diff > > > Do you have output for this? > > I did this with 1), and this is the log. Thanks for the log, it helps. Can you please boot the same kernel (the one that hangs) with the `nosmp' option passed to Xen? ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Improve presentation of objcopy output
This makes it easier to grep log files, and just looks a lot better in the console output of a make. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- Makefile |6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff -r a41b8cd01ad2 xen/arch/powerpc/Makefile --- a/xen/arch/powerpc/Makefile Fri Oct 06 12:15:06 2006 -0500 +++ b/xen/arch/powerpc/Makefile Fri Oct 06 17:45:51 2006 -0400 @@ -119,11 +119,9 @@ xen-syms.o: xen-syms.S $(CC) $(CFLAGS) $(TARGET_OPTS) xen-syms.o -o $@ # our firmware only loads 32-bit ELF files +OCPYFLAGS= --input-target=elf64-powerpc --output-target=elf32-powerpc $(TARGET): $(TARGET)-syms - $(CROSS_COMPILE)objcopy \ - --input-target=elf64-powerpc \ - --output-target=elf32-powerpc \ - $^ $@ + $(CROSS_COMPILE)objcopy $(OCPYFLAGS) $^ $@ asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c $(HDRS) $(CC) $(CFLAGS) -S -o $@ $< ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Cannot boot from local disk
I took a look at the situation. It boils down to this: 1. In the internal Xen tree, there is a bug (my fault) in which firmware bootargs are overridden by the builtin "xen" bootarg. This is fixed (by Hollis) in the current public Xen tree. 2. In the internal Xen tree, the SMP patch apparently causes dom0 Linux to fail on Kawachiya's JS20, but no other JS20. 3. In the public Xen tree, the 32-bit objcopy produces an image that Kawachiya's yaboot apparently cannot load. The catch-22 is that if Maria updates the internal tree to get the fixes in the public tree, the bootargs problem will go away, but the yaboot problem will appear. So we really need to know what yaboot is complaining about. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Rolled up bootargs simplification
On Wed, Oct 04, 2006 at 05:34:05PM -0500, Hollis Blanchard wrote: > On Mon, 2006-10-02 at 11:46 -0400, Amos Waterland wrote: > > This patch has been in use by an internal IBM project for some time. > > It enables two important things: allowing developers and cluster > > administrators the option of overriding the bootargs supplied by > > firmware, and the ability to take a single gold master xen binary and > > customize its bootargs across a cluster with a simple and well-tested > > post-processing tool. > > Thanks for your patience Amos; I believe the commits I just made should > solve your problem. Basically removing the boot wrapper (and that's > bugged me for a while for other reasons) allows us to collapse three > command line buffers into one, and that makes me happy. :) > > bicl (http://www.cs.unm.edu/~k42/tools/bicl) acts directly on the > command line that was built in at compile time. If that buffer is empty > at runtime, firmware arguments will be used. > > Note that changing the CMDLINE at compile time does not properly replace > the previous CMDLINE; you would have to make clean or remove > xen/arch/powerpc/cmdline.o by hand. I'd happily accept a Makefile patch > for that. I have tested the commits pretty heavily and things work fine. Thanks. I am doing automated reliability runs now like this: $ bicl xen "xen dom0_mem=%M --" $ for i in 128M 192M 1G 2G 7G; do cp xen xen.$i && bicl xen.$i "$(bicl xen | sed "s,%M,$i,")" && netboot-js21-blade -f xen.$i; done ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [PATCH] Re: [XenPPC] XenPPC: redundancy definition of __trap_to_gdb with CRASH_DEBUG
On Thu, Oct 05, 2006 at 01:52:39PM +1000, Tony Breeds wrote: > On Wed, Oct 04, 2006 at 07:36:08AM -0400, Jimi Xenidis wrote: > > I think hollis nailed correctly, does it work for everyone? > > Works for me. Did you try building with debug=y, then with debug=n, with no intervening `make clean'? ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Builtin cmdline dependency rule
Rebuild cmdline.o when the user changes the CMDLINE=X argument passed to the make invocation. I couldn't find an example of another project that handles this case properly, so I came up with this. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- .hgignore |1 + xen/arch/powerpc/Makefile | 16 +++- 2 files changed, 16 insertions(+), 1 deletion(-) diff -r 1b759b2522cd .hgignore --- a/.hgignore Wed Oct 04 17:27:16 2006 -0500 +++ b/.hgignore Wed Oct 04 20:26:04 2006 -0400 @@ -207,6 +207,7 @@ ^xen/arch/powerpc/xen\.lds$ ^xen/arch/powerpc/.xen-syms$ ^xen/arch/powerpc/xen-syms.S$ +^xen/arch/powerpc/cmdline.dep$ ^unmodified_drivers/linux-2.6/\.tmp_versions ^unmodified_drivers/linux-2.6/.*\.cmd$ ^unmodified_drivers/linux-2.6/.*\.ko$ diff -r 1b759b2522cd xen/arch/powerpc/Makefile --- a/xen/arch/powerpc/Makefile Wed Oct 04 17:27:16 2006 -0500 +++ b/xen/arch/powerpc/Makefile Wed Oct 04 20:26:24 2006 -0400 @@ -80,6 +80,20 @@ ifneq ($(CMDLINE),) # The first token in the arguments will be silently dropped. FULL_CMDLINE := xen $(CMDLINE) endif + +ifeq ($(wildcard cmdline.dep),) +cmdline.dep: + echo $(FULL_CMDLINE) > cmdline.dep +else +ifneq ($(FULL_CMDLINE),$(shell cat cmdline.dep)) +cmdline.dep:: + echo $(FULL_CMDLINE) > cmdline.dep +else +cmdline.dep: +endif +endif + +cmdline.o: cmdline.dep cmdline.o: CFLAGS += -DCMDLINE="\"$(FULL_CMDLINE)\"" TARGET_OPTS = $(OMAGIC) -Wl,-Ttext,0x40,-T,xen.lds @@ -122,4 +136,4 @@ dom0.bin: $(DOM0_IMAGE) clean:: $(MAKE) -f $(BASEDIR)/Rules.mk -C of_handler clean - rm -f firmware firmware_image.bin dom0.bin .xen-syms + rm -f firmware firmware_image.bin dom0.bin .xen-syms cmdline.dep ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] XenPPC: redundancy definition of __trap_to_gdb with CRASH_DEBUG
On Tue, Oct 03, 2006 at 02:53:02PM -0400, Yi Ge wrote: > It seems the gcc couldn't find the prototype of __trap_to_gdb when make the > function call in debugger_trap_fatal(). So I added an extern prototype here > with ifndef macro. I'm not positive of this, but I'm pretty sure that this is caused by improper dependency checking. I suspect that you changed from debug=n to debug=y and got this. I'd suggest doing a `make clean' and trying again without your patch. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Re: Automated reliability report for SMP patch on JS2x
On Tue, Oct 03, 2006 at 12:25:33PM -0400, Maria Butrico wrote: > What's really interesting to me about this is that the invocation of the > icache invalidation did not go in till later. So if anything we could find > this to be even more reliable one the other changes are also picked up. The key commit from my perspective was the flush of the icache on secondary processor spinup. That made SMP spinup on JS2x quite solid, in my experiments. The function you are talking about is relevant when destroying a domain and loading a new one, I believe. Note that the tests reported in this email only create domains, because I knew that the invocation you are talking about had not gone in yet. > I missed this: what is transient? Transient is how my tool classifies things like the network going down for five minutes at 3:00 a.m., which causes TFTP to fail. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Automated reliability report for SMP patch on JS2x
An automated process has boooted Xen on two JS21 blades and one JS20 blade a total of 1241 times, recording 0 failures and 1237 passes, using a correctness criteria of the creation of four domU's for the first JS21, the launch of the ssh daemon for the second JS21, and the creation of two domU's for the JS20. The version of Xen used was changeset dd95dd13cd3e+, which is the following tip of tree changeset plus the bootargs simplification patch plus the SMP patch: changeset: 12184:02f6e775deb1 user:Jimi Xenidis <[EMAIL PROTECTED]> date:Mon Oct 02 11:07:54 2006 -0400 summary: Add Function to completely flush the I-Cache for a processor --- changeset : dd95dd13cd3e+ machine : cso92 pass: 401 fail: 0 transient : 3 total : 404 reliability : 100.0% changeset : dd95dd13cd3e+ machine : kpblade7 pass: 423 fail: 0 transient : 1 total : 424 reliability : 100.0% changeset : dd95dd13cd3e+ machine : kpblade1 pass: 413 fail: 0 transient : 0 total : 413 reliability : 100.0% ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Rolled up bootargs simplification
Merge in Jimi's idea of handling the seperator phrase entirely in boot_of.
Set and document clear precedence rules for boot argument processing.
Support the same builtin command line format in the 32-bit xen binary as
the 32-bit zImage binary does. Remove dead default_bootargs code.
This patch has been in use by an internal IBM project for some time. It
enables two important things: allowing developers and cluster
administrators the option of overriding the bootargs supplied by
firmware, and the ability to take a single gold master xen binary and
customize its bootargs across a cluster with a simple and well-tested
post-processing tool.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
b/xen/arch/powerpc/boot/arg32.c | 22 ++
xen/arch/powerpc/Makefile |5 -
xen/arch/powerpc/boot/boot32.S |5 +
xen/arch/powerpc/boot_of.c | 24 +++-
xen/arch/powerpc/ofd_fixup.c| 19 ---
xen/arch/powerpc/xen.lds.S |4
6 files changed, 54 insertions(+), 25 deletions(-)
diff -r dbfb5fc0b9b2 xen/arch/powerpc/Makefile
--- a/xen/arch/powerpc/Makefile Fri Sep 29 11:29:32 2006 -0400
+++ b/xen/arch/powerpc/Makefile Thu Sep 28 14:41:54 2006 -0400
@@ -130,7 +130,10 @@ boot32.o: boot/boot32.S
$(CC) -m32 -Wa,-a32,-mppc64bridge \
-D__ASSEMBLY__ -D__BRIDGE64__ $(CFLAGS) -c $< -o $@
-$(TARGET): boot32.o $(TARGET).bin.o
+arg32.o: boot/arg32.c
+ $(CC) -m32 -DCMDLINE="\"$(IMAGENAME) $(CMDLINE)\"" -c $< -o $@
+
+$(TARGET): boot32.o arg32.o $(TARGET).bin.o
$(CC) -m32 -N -Wl,-melf32ppclinux -static -nostdlib \
-Wl,-Ttext,$(boot32_link_base) -Wl,-Tdata,$(xen_link_base) \
$(CFLAGS) $^ -o $@
diff -r dbfb5fc0b9b2 xen/arch/powerpc/boot/boot32.S
--- a/xen/arch/powerpc/boot/boot32.SFri Sep 29 11:29:32 2006 -0400
+++ b/xen/arch/powerpc/boot/boot32.SThu Sep 28 14:41:54 2006 -0400
@@ -36,6 +36,11 @@ _start:
## 51 12 1000 ME Machine Check Enable
_real_start:
+ # pass the builtin command line as argument to hype_init
+ li 7, 0
+ oris7, 7, [EMAIL PROTECTED]
+ ori 7, 7, [EMAIL PROTECTED]
+
# pass the original msr as argument to hype_init
mfmsr 8
diff -r dbfb5fc0b9b2 xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cFri Sep 29 11:29:32 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cMon Oct 02 11:37:04 2006 -0400
@@ -40,11 +40,12 @@ static ulong of_vec;
static ulong of_vec;
static ulong of_msr;
static int of_out;
-static char bootargs[256];
#define COMMAND_LINE_SIZE 512
static char builtin_cmdline[COMMAND_LINE_SIZE]
__attribute__((section("__builtin_cmdline"))) = CMDLINE;
+
+static char bootargs[COMMAND_LINE_SIZE];
extern struct ns16550_defaults ns16550;
@@ -460,13 +461,24 @@ static void boot_of_probemem(multiboot_i
}
}
-static void boot_of_bootargs(multiboot_info_t *mbi)
+static void
+boot_of_bootargs(multiboot_info_t *mbi, char *wrapper_builtin_cmdline)
{
int rc;
-rc = of_getprop(bof_chosen, "bootargs", &bootargs, sizeof (bootargs));
-if (rc == OF_FAILURE || bootargs[0] == '\0') {
+/* Boot argument precedence rules:
+ * 1. Arguments from 32-bit wrapper override all else
+ * 2. Builtin arguments in 64-bit image override firmware
+ * 3. Firmware is used if neither of the above exist
+ */
+if (wrapper_builtin_cmdline[0] != 0) {
+strlcpy(bootargs, wrapper_builtin_cmdline, sizeof(bootargs));
+} else if (builtin_cmdline[0] != 0) {
strlcpy(bootargs, builtin_cmdline, sizeof(bootargs));
+} else {
+rc = of_getprop(bof_chosen, "bootargs", &bootargs, sizeof (bootargs));
+if (rc > sizeof (bootargs))
+of_panic("bootargs[] not big enough for /chosen/bootargs\n");
}
mbi->flags |= MBI_CMDLINE;
@@ -1023,6 +1035,8 @@ static void * __init boot_of_module(ulon
mods[mod].mod_start, mods[mod].mod_end);
p = strstr((char *)(ulong)mbi->cmdline, sepr);
if (p != NULL) {
+/* Xen proper should never know about the dom0 args. */
+*(char *)p = '\0';
p += sizeof (sepr) - 1;
mods[mod].string = (u32)(ulong)p;
of_printf("%s: dom0 mod string: %s\n", __func__, p);
@@ -1197,7 +1211,7 @@ multiboot_info_t __init *boot_of_init(
boot_of_fix_maple();
boot_of_probemem(&mbi);
-boot_of_bootargs(&mbi);
+boot_of_bootargs(&mbi, (char *)r7);
oft = boot_of_module(r3, r4, &mbi);
boot_of_cpus();
boot_of_serial(oft);
diff -r dbfb5fc0b9b2 xen/arch/powerpc/ofd_fixup.c
--- a/xen/arch/powerpc/ofd_fixup.c Fri Sep 29 11:29:32 2006 -0400
+++ b/xen/arch/powerpc/ofd_fixup.c Thu Sep 28 14:41:54 2006 -0400
@@ -244,12 +244,6 @@ stat
[XenPPC] [PATCH] Parse Xen command line properly
We are improperly feeding the entire boot parameter string to Xen's
generic command line parser. This can have unexpected results when one
of the dom0 parameters, such as console=X, has meaning to the Xen
parser. First reported by Maria Butrico.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
setup.c | 16 ++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff -r 261c458e46af xen/arch/powerpc/setup.c
--- a/xen/arch/powerpc/setup.c Fri Sep 29 18:13:27 2006 -0400
+++ b/xen/arch/powerpc/setup.c Mon Oct 02 00:17:07 2006 -0400
@@ -279,8 +279,20 @@ static void __init __start_xen(multiboot
ticks_per_usec = timebase_freq / 100ULL;
/* Parse the command-line options. */
-if ((mbi->flags & MBI_CMDLINE) && (mbi->cmdline != 0))
-cmdline_parse(__va((ulong)mbi->cmdline));
+if ((mbi->flags & MBI_CMDLINE) && (mbi->cmdline != 0)) {
+char *end, *src = (char *)(ulong)mbi->cmdline;
+char buff[256];
+
+end = strstr(src, "--");
+
+if (end && (end - src < sizeof(buff))) {
+strlcpy(buff, src, end - src);
+} else {
+strlcpy(buff, src, sizeof(buff));
+}
+
+cmdline_parse(buff);
+}
/* We initialise the serial devices very early so we can get debugging. */
ns16550.io_base = 0x3f8;
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] Document boot argument processing
Hollis requested that boot argument processing be documented. I have placed the following text on this wiki: http://wiki.xensource.com/xenwiki/XenPPC/BootArguments --- The boot argument processing for powerpc Xen is much less complex than it may appear. Assuming that the 'Simplify bootargs processing' patch is accepted in some form, the following rules apply: 1. Arguments from 32-bit wrapper override all else 2. Builtin arguments in 64-bit image override firmware 3. Firmware is used if neither of the above exist It is true that dom0 Linux has its own boot argument processing, but there is a simple transitive relation between Xen's bootargs and Linux's bootargs. That is, Xen first applies the above three rules to decide what to feed Linux. Linux then applies its three rules (it orders their precedence slightly differently) to decide what to feed the kernel proper. The format of the builtin command line buffer is essentially just a seperate ELF section, which makes it easy for post-processing tools to locate it and edit it. If you have a 2.6.17 or later zImage kernel available, the following is illustrative: $ objdump -h arch/powerpc/boot/zImage | grep cmdline 2 __builtin_cmdline 0200 00406274 00406274 00016274 2**2 The format of the builtin command line buffer is simple, so many tools may one day understand it, but at present the most commonly-used tool is `bicl', which is shipped under the LGPL as part of the K42 source: http://www.cs.unm.edu/~k42/tools/bicl The usage of the tool is simple: $ bicl `builtin-cmdline' manipulates builtin command line arguments Usage: builtin-cmdline FILE [ARGS] -?, --help Show this help statement. --version Show version statement. Examples: builtin-cmdline zImage builtin-cmdline zImage 'console=ttyS0' The tool is used every day to edit the builtin command lines of Linux and Xen binaries as part of an internal IBM project. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Simplify bootargs processing
Set and document clear precedence rules for boot argument processing.
Support the same builtin command line format in the 32-bit xen binary as
the 32-bit zImage binary does. Remove dead default_bootargs code.
This patch has been in use by an internal IBM project for some time. It
enables two important things: allowing developers and cluster
administrators the option of overriding the bootargs supplied by
firmware, and the ability to take a single gold master xen binary and
customize its bootargs across a cluster with a simple and well-tested
post-processing tool.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
b/xen/arch/powerpc/boot/arg32.c | 22 ++
xen/arch/powerpc/Makefile |5 -
xen/arch/powerpc/boot/boot32.S |5 +
xen/arch/powerpc/boot_of.c | 17 +
xen/arch/powerpc/ofd_fixup.c| 19 ---
xen/arch/powerpc/xen.lds.S |4
6 files changed, 48 insertions(+), 24 deletions(-)
diff -r c9bf3af5624b xen/arch/powerpc/Makefile
--- a/xen/arch/powerpc/Makefile Thu Sep 28 12:34:01 2006 -0400
+++ b/xen/arch/powerpc/Makefile Thu Sep 28 14:41:54 2006 -0400
@@ -130,7 +130,10 @@ boot32.o: boot/boot32.S
$(CC) -m32 -Wa,-a32,-mppc64bridge \
-D__ASSEMBLY__ -D__BRIDGE64__ $(CFLAGS) -c $< -o $@
-$(TARGET): boot32.o $(TARGET).bin.o
+arg32.o: boot/arg32.c
+ $(CC) -m32 -DCMDLINE="\"$(IMAGENAME) $(CMDLINE)\"" -c $< -o $@
+
+$(TARGET): boot32.o arg32.o $(TARGET).bin.o
$(CC) -m32 -N -Wl,-melf32ppclinux -static -nostdlib \
-Wl,-Ttext,$(boot32_link_base) -Wl,-Tdata,$(xen_link_base) \
$(CFLAGS) $^ -o $@
diff -r c9bf3af5624b xen/arch/powerpc/boot/boot32.S
--- a/xen/arch/powerpc/boot/boot32.SThu Sep 28 12:34:01 2006 -0400
+++ b/xen/arch/powerpc/boot/boot32.SThu Sep 28 14:41:54 2006 -0400
@@ -36,6 +36,11 @@ _start:
## 51 12 1000 ME Machine Check Enable
_real_start:
+ # pass the builtin command line as argument to hype_init
+ li 7, 0
+ oris7, 7, [EMAIL PROTECTED]
+ ori 7, 7, [EMAIL PROTECTED]
+
# pass the original msr as argument to hype_init
mfmsr 8
diff -r c9bf3af5624b xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cThu Sep 28 12:34:01 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cThu Sep 28 14:41:54 2006 -0400
@@ -460,13 +460,22 @@ static void boot_of_probemem(multiboot_i
}
}
-static void boot_of_bootargs(multiboot_info_t *mbi)
+static void
+boot_of_bootargs(multiboot_info_t *mbi, char *wrapper_builtin_cmdline)
{
int rc;
-rc = of_getprop(bof_chosen, "bootargs", &bootargs, sizeof (bootargs));
-if (rc == OF_FAILURE || bootargs[0] == '\0') {
+/* Boot argument precedence rules:
+ * 1. Arguments from 32-bit wrapper override all else
+ * 2. Builtin arguments in 64-bit image override firmware
+ * 3. Firmware is used if neither of the above exist
+ */
+if (wrapper_builtin_cmdline[0] != 0) {
+strlcpy(bootargs, wrapper_builtin_cmdline, sizeof(bootargs));
+} else if (builtin_cmdline[0] != 0) {
strlcpy(bootargs, builtin_cmdline, sizeof(bootargs));
+} else {
+rc = of_getprop(bof_chosen, "bootargs", &bootargs, sizeof (bootargs));
}
mbi->flags |= MBI_CMDLINE;
@@ -1197,7 +1206,7 @@ multiboot_info_t __init *boot_of_init(
boot_of_fix_maple();
boot_of_probemem(&mbi);
-boot_of_bootargs(&mbi);
+boot_of_bootargs(&mbi, (char *)r7);
oft = boot_of_module(r3, r4, &mbi);
boot_of_cpus();
boot_of_serial(oft);
diff -r c9bf3af5624b xen/arch/powerpc/ofd_fixup.c
--- a/xen/arch/powerpc/ofd_fixup.c Thu Sep 28 12:34:01 2006 -0400
+++ b/xen/arch/powerpc/ofd_fixup.c Thu Sep 28 14:41:54 2006 -0400
@@ -244,12 +244,6 @@ static ofdn_t ofd_xics_props(void *m)
}
#endif
-/*
- * Good things you can stick here:
- * init=/bin/bash ip=dhcp root=/dev/hda2 ide=nodma
- */
-static char default_bootargs[] = "";
-
static ofdn_t ofd_chosen_props(void *m, const char *cmdline)
{
ofdn_t n;
@@ -257,7 +251,6 @@ static ofdn_t ofd_chosen_props(void *m,
static const char path[] = "/chosen";
char bootargs[256];
int bsz;
-int sz;
int rm;
n = ofd_node_find(m, path);
@@ -270,18 +263,6 @@ static ofdn_t ofd_chosen_props(void *m,
strcpy(bootargs, cmdline);
bsz = strlen(bootargs) + 1;
rm = sizeof (bootargs) - bsz;
-
-if (default_bootargs != NULL) {
-sz = strlen(default_bootargs);
-if (sz > rm) {
-panic("default_bootargs is too big: 0x%x > 0x%x\n",
- sz, rm);
-} else if (sz > 0) {
-memcpy(&bootargs[bsz - 1], default_bootargs, sz + 1);
-bsz += sz;
-rm -= sz;
-}
-
[XenPPC] Automated reliability report for Xen on JS20
An automated process boooted Xen on a JS20 blade 534 times, recording 0 failures and 534 passes, using the launch of the ssh daemon by dom0 as a correctness criteria. The version of Xen used was changeset 261c458e46af, which is tip of tree changeset dbfb5fc0b9b2 plus the bootargs simplification patch. --- changeset : 261c458e46af machine : kpblade1 pass: 534 fail: 0 transient : 0 total : 534 reliability : 100.0% ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Automated reliability report for Xen on JS21
An automated process boooted Xen on a JS21 blade 126 times, recording 5 failures and 109 passes, using the launch of the ssh daemon by dom0 as a correctness criteria. The version of Xen used was changeset 261c458e46af, which is tip of tree changeset dbfb5fc0b9b2 plus the bootargs simplification patch. --- changeset : 261c458e46af machine : cso92 pass: 109 fail: 5 transient : 12 total : 126 reliability : 96.0% 261c458e46af/cso92/xen.log.2006-09-29.204918 (XEN) Final offset: 5 (3/300) (XEN) Synchronizing timebase (XEN) 261c458e46af/cso92/xen.log.2006-09-29.214807 (XEN) Final offset: 4 (265/300) (XEN) Synchronizing timebase (XEN) 261c458e46af/cso92/xen.log.2006-09-30.123950 (XEN) Final offset: 5 (39/300) (XEN) Synchronizing timebase (XEN) 261c458e46af/cso92/xen.log.2006-09-30.130239 (XEN) Final offset: 6 (53/300) (XEN) Synchronizing timebase (XEN) 261c458e46af/cso92/xen.log.2006-09-30.135944 (XEN) Final offset: 5 (59/300) (XEN) Synchronizing timebase (XEN) ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix compile error in gdbstub.c
With crash_debug=y the compiler can't find a prototype for unimplemented(),
where gdb-arch-read_reg is calling it.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
gdbstub.c |1 +
1 file changed, 1 insertion(+)
diff -r 0e83ba62a3a5 xen/arch/powerpc/gdbstub.c
--- a/xen/arch/powerpc/gdbstub.cFri Sep 22 11:51:56 2006 -0400
+++ b/xen/arch/powerpc/gdbstub.cFri Sep 22 14:19:11 2006 -0400
@@ -25,6 +25,7 @@
#include
#include
#include
+#include
#include
asm(".globl trap_instruction\n"
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Print backtrace on BUG
This makes Xen/PPC dump a backtrace when a BUG() is triggered.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
config.h | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff -r 5418062d2da8 xen/include/asm-powerpc/powerpc64/config.h
--- a/xen/include/asm-powerpc/powerpc64/config.h
+++ b/xen/include/asm-powerpc/powerpc64/config.h
@@ -38,7 +38,15 @@
#ifndef __ASSEMBLY__
-#define FORCE_CRASH() __asm__ __volatile__ ( "trap" )
+#define FORCE_CRASH() \
+do { \
+unsigned long sp, lr, pc, tp; \
+asm("mr %0, 1" : "=r" (sp)); \
+asm("mflr %0" : "=r" (lr)); \
+asm("mflr %0; bl 1f; 1: mflr %1; mtlr %0" : "=r" (tp), "=r" (pc));\
+show_backtrace(sp, lr, pc); \
+__asm__ __volatile__ ( "trap" ); \
+} while ( 0 )
#endif /* __ASSEMBLY__ */
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Assertion 'entry->next->prev == entry' failed
On Wed, Sep 20, 2006 at 10:52:32PM -0400, Amos Waterland wrote: > On Wed, Sep 20, 2006 at 07:20:54PM -0400, Amos Waterland wrote: > > On Wed, Sep 20, 2006 at 11:35:51AM -0500, Hollis Blanchard wrote: > > > On Tue, 2006-09-19 at 20:04 -0400, Amos Waterland wrote: > > > > I am seeing this when loading dom0 now on JS21: > > > > > > > > (XEN) *** LOADING DOMAIN 0 *** > > > > (XEN) Assertion 'entry->next->prev == entry' failed, line 92 ... > > > > (XEN) BUG at /home/apw/devel/xen/xen.hg/xen/include/xen/list.h:92 > > > > > > > > I am using latest Xen and Linux. > > > > > > I cannot reproduce; I successfully started three domUs before exhausting > > > my Maple's memory. > > > > It seems that this bug only fires on JS21's running the latest > > SLOF. I can boot the exact same binary image of Xen and Linux on a JS21 > > running the old SLOF and things work fine. > > I take this back. I flashed a known-good image of SLOF on the new JS21 > and the same assertion fires. > > This JS21 has 8GB of RAM, and I think that may be exposing a problem > in SLOF or Xen. Here is the backtrace: (XEN) Assertion 'entry->next->prev == entry' failed, line 92, file /home/apw/devel/xen/xen.hg/xen/include/xen/list.h (XEN) BUG at /home/apw/devel/xen/xen.hg/xen/include/xen/list.h:92 (XEN) [004639A0] 00431BC8 .__bug+0x74/0x8c (unreliable) (XEN) [00463A40] 0041738C .list_del+0x6c/0x104 (XEN) [00463AC0] 00417044 .alloc_heap_pages+0x140/0x28c (XEN) [00463B60] 00418990 .alloc_domheap_pages+0x14c/0x4ec (XEN) [00463C40] 00441924 .allocate_extents+0x74/0x108 (XEN) [00463CF0] 00438E64 .construct_dom0+0x298/0xbcc (XEN) [00463E90] 004581F0 .__start_xen+0x56c/0x640 (XEN) [00463F70] 00458360 .__start_xen_ppc+0x9c/0xbc (XEN) [00464000] 00400138 _start+0x38/0x3c ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Assertion 'entry->next->prev == entry' failed
On Wed, Sep 20, 2006 at 07:20:54PM -0400, Amos Waterland wrote: > On Wed, Sep 20, 2006 at 11:35:51AM -0500, Hollis Blanchard wrote: > > On Tue, 2006-09-19 at 20:04 -0400, Amos Waterland wrote: > > > I am seeing this when loading dom0 now on JS21: > > > > > > (XEN) *** LOADING DOMAIN 0 *** > > > (XEN) Assertion 'entry->next->prev == entry' failed, line 92 ... > > > (XEN) BUG at /home/apw/devel/xen/xen.hg/xen/include/xen/list.h:92 > > > > > > I am using latest Xen and Linux. > > > > I cannot reproduce; I successfully started three domUs before exhausting > > my Maple's memory. > > It seems that this bug only fires on JS21's running the latest > SLOF. I can boot the exact same binary image of Xen and Linux on a JS21 > running the old SLOF and things work fine. I take this back. I flashed a known-good image of SLOF on the new JS21 and the same assertion fires. This JS21 has 8GB of RAM, and I think that may be exposing a problem in SLOF or Xen. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Assertion 'entry->next->prev == entry' failed
On Wed, Sep 20, 2006 at 11:35:51AM -0500, Hollis Blanchard wrote: > On Tue, 2006-09-19 at 20:04 -0400, Amos Waterland wrote: > > I am seeing this when loading dom0 now on JS21: > > > > (XEN) *** LOADING DOMAIN 0 *** > > (XEN) Assertion 'entry->next->prev == entry' failed, line 92 ... > > (XEN) BUG at /home/apw/devel/xen/xen.hg/xen/include/xen/list.h:92 > > > > I am using latest Xen and Linux. > > I cannot reproduce; I successfully started three domUs before exhausting > my Maple's memory. It seems that this bug only fires on JS21's running the latest SLOF. I can boot the exact same binary image of Xen and Linux on a JS21 running the old SLOF and things work fine. I will follow up with the SLOF people on this. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Assertion 'entry->next->prev == entry' failed
I am seeing this when loading dom0 now on JS21: (XEN) *** LOADING DOMAIN 0 *** (XEN) Assertion 'entry->next->prev == entry' failed, line 92, file /home/apw/devel/xen/xen.hg/xen/include/xen/list.h (XEN) BUG at /home/apw/devel/xen/xen.hg/xen/include/xen/list.h:92 I am using latest Xen and Linux. On Thu, Sep 14, 2006 at 06:21:06PM -0400, Amos Waterland wrote: > With current linux.hg and xen.hg, I am getting this consistently when > creating my third domU: > > (XEN) Domain[2].0: initializing > (XEN) Assertion 'entry->next->prev == entry' failed, line 92, file > xen/include/xen/list.h > (XEN) BUG at xen/include/xen/list.h:92 > > > ___ > Xen-ppc-devel mailing list > Xen-ppc-devel@lists.xensource.com > http://lists.xensource.com/xen-ppc-devel ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Option to override firmware bootargs
On Mon, Sep 18, 2006 at 09:51:05AM -0500, Hollis Blanchard wrote: > I'm happy to accommodate netbooting large clusters, but I'm asking you > to simplify this situation, not make it more complex. In order to scale to large clusters, one must have flexible and policy-free software. Right now Xen has a certain policy hard-coded in it: namely, that the firmware-supplied bootargs always take precedence over the builtin bootargs with no option to change. This patch is three lines of code, makes Xen more flexible, removes a hard-coded policy, and solves a real-world problem. Seen in that light, I hope you will reconsider your objection. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH] Option to override firmware bootargs
On Mon, Sep 18, 2006 at 09:19:29AM -0500, Hollis Blanchard wrote: > On Sun, 2006-09-17 at 02:40 -0400, Amos Waterland wrote: > > I recently got a few hours on a Maple development board. I wanted to > > boot with an nfsroot, but I didn't want to risk changing the bootargs > > saved in the PIBS firmware. This patch lets Xen choose to override the > > bootargs supplied by the firmware if the user specifies `bootargs=builtin'. > > We already have too many ways of passing boot parameters. I'll be taking > a look at GRUB in the near future, so hopefully most of them will become > obsolete. This is not a very good response. Perhaps I did not properly explain what the problem is. The patch solves a real problem that using GRUB does not. I need to boot Xen on a large number of machines (50 or more), and whenever you reach that scale you find it necessary to reduce the state of each actor in the system. I want to reduce the state saved on each individual machine. I.e, if the last person who happened to work on a particular blade happened to put a boot argument in the firmware that breaks me, I want to be able to override him. I need to boot 50 or more machines at once and I do not control the machines, I only get access to them periodically. Please explain how using GRUB is going to help. Keep in mind that I might not be able to force the machine owners to install GRUB, and many of the machines don't even have a hard drive. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Option to override firmware bootargs
I recently got a few hours on a Maple development board. I wanted to
boot with an nfsroot, but I didn't want to risk changing the bootargs
saved in the PIBS firmware. This patch lets Xen choose to override the
bootargs supplied by the firmware if the user specifies `bootargs=builtin'.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
boot_of.c |5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff -r 1d9c135673e4 xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cFri Sep 15 18:20:55 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cSun Sep 17 02:02:51 2006 -0400
@@ -455,9 +455,12 @@ static void boot_of_bootargs(multiboot_i
static void boot_of_bootargs(multiboot_info_t *mbi)
{
int rc;
+char *override = NULL;
+
+override = strstr(builtin_cmdline, "bootargs=builtin");
rc = of_getprop(bof_chosen, "bootargs", &bootargs, sizeof (bootargs));
-if (rc == OF_FAILURE || bootargs[0] == '\0') {
+if (rc == OF_FAILURE || bootargs[0] == '\0' || override) {
strlcpy(bootargs, builtin_cmdline, sizeof(bootargs));
}
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] SMP support
Enable secondary processors as schedulable entities and allow domU's to
run on them. Tested extensively on JS20 and JS21 with up to ten
domains at a time.
Note that this patch relies on the hdec storm patch just posted.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
setup.c | 36
1 file changed, 16 insertions(+), 20 deletions(-)
diff -r c94df1e4e62c xen/arch/powerpc/setup.c
--- a/xen/arch/powerpc/setup.c Thu Sep 14 15:43:38 2006 -0500
+++ b/xen/arch/powerpc/setup.c Thu Sep 14 18:37:30 2006 -0400
@@ -89,6 +89,8 @@ extern char __per_cpu_start[], __per_cpu
/* move us to a header file */
extern void initialize_keytable(void);
+static struct domain *idle_domain;
+
volatile struct processor_area * volatile global_cpu_table[NR_CPUS];
int is_kernel_text(unsigned long addr)
@@ -152,8 +154,6 @@ static void percpu_free_unused_areas(voi
static void __init start_of_day(void)
{
-struct domain *idle_domain;
-
init_IRQ();
scheduler_init();
@@ -167,23 +167,6 @@ static void __init start_of_day(void)
/* for some reason we need to set our own bit in the thread map */
cpu_set(0, cpu_sibling_map[0]);
-
-percpu_free_unused_areas();
-
-{
-/* FIXME: Xen assumes that an online CPU is a schedualable
- * CPU, but we just are not there yet. Remove this fragment when
- * scheduling processors actually works. */
-int cpuid;
-
-printk("WARNING!: Taking all secondary CPUs offline\n");
-
-for_each_online_cpu(cpuid) {
-if (cpuid == 0)
-continue;
-cpu_clear(cpuid, cpu_online_map);
-}
-}
initialize_keytable();
/* Register another key that will allow for the the Harware Probe
@@ -257,9 +240,19 @@ static int kick_secondary_cpus(int maxcp
/* This is the first C code that secondary processors invoke. */
int secondary_cpu_init(int cpuid, unsigned long r4)
{
+struct vcpu *vcpu;
+
cpu_initialize(cpuid);
smp_generic_take_timebase();
cpu_set(cpuid, cpu_online_map);
+
+vcpu = alloc_vcpu(idle_domain, cpuid, cpuid);
+BUG_ON(vcpu == NULL);
+
+set_current(idle_domain->vcpu[cpuid]);
+idle_vcpu[cpuid] = current;
+startup_cpu_idle_loop();
+
while(1);
}
@@ -332,6 +325,8 @@ static void __init __start_xen(multiboot
debugger_trap_immediate();
#endif
+start_of_day();
+
/* Deal with secondary processors. */
if (opt_nosmp || ofd_boot_cpu == -1) {
printk("nosmp: leaving secondary processors spinning forever\n");
@@ -340,7 +335,8 @@ static void __init __start_xen(multiboot
kick_secondary_cpus(max_cpus);
}
-start_of_day();
+/* This cannot be called before secondary cpus are marked online. */
+percpu_free_unused_areas();
/* Create initial domain 0. */
dom0 = domain_create(0);
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix infinite loop caused by hdec storm
This was the cause of the periodic hang on secondary processors that has
been holding back the submission of the SMP patch.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
time.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff -r c94df1e4e62c xen/arch/powerpc/time.c
--- a/xen/arch/powerpc/time.c Thu Sep 14 15:43:38 2006 -0500
+++ b/xen/arch/powerpc/time.c Thu Sep 14 18:28:24 2006 -0400
@@ -60,7 +60,7 @@ int reprogram_timer(s_time_t timeout)
s_time_t expire;
if (timeout == 0) {
-expire = 0;
+expire = INT_MAX;
} else {
s_time_t now;
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Improved tracing facility
This is not a real tracing facility, but it is good enough to find bugs.
It supports symbolic lookup and tracepoint insertion, as well as hooking
the three main entry points to the hypervisor.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/exceptions.c| 79 +++
arch/powerpc/external.c | 20 ++
arch/powerpc/hcalls.c| 20 ++
include/asm-powerpc/config.h | 11 +
4 files changed, 130 insertions(+)
diff -r c94df1e4e62c xen/arch/powerpc/exceptions.c
--- a/xen/arch/powerpc/exceptions.c Thu Sep 14 15:43:38 2006 -0500
+++ b/xen/arch/powerpc/exceptions.c Thu Sep 14 18:33:35 2006 -0400
@@ -29,6 +29,40 @@
#undef DEBUG
+#ifdef TRACE_DOMAINS
+long iars[MAX_IARS][2];
+long last_dump_time = 0;
+int cpu_to_trace = 3;
+
+void trace_function(void *addr)
+{
+long iar = ((long *)(addr))[0];
+
+return trace_address(iar);
+}
+
+void trace_address(long iar)
+{
+int i;
+
+if (smp_processor_id() != cpu_to_trace)
+ return;
+
+for (i = 0; iars[i][0] != 0 && i < MAX_IARS; i++) {
+ if (iars[i][0] == iar) {
+ iars[i][1]++;
+ i = MAX_IARS;
+ break;
+ }
+}
+
+if (i < MAX_IARS) {
+ iars[i][0] = iar;
+ iars[i][1] = 1;
+}
+}
+#endif
+
extern ulong ppc_do_softirq(ulong orig_msr);
extern void do_timer(struct cpu_user_regs *regs);
extern void do_dec(struct cpu_user_regs *regs);
@@ -41,6 +75,51 @@ void do_timer(struct cpu_user_regs *regs
/* Set HDEC high so it stops firing and can be reprogrammed by
* set_preempt() */
mthdec(INT_MAX);
+
+#ifdef TRACE_DOMAINS
+if (smp_processor_id() == cpu_to_trace) {
+ int i;
+ long iar = regs->pc;
+
+ for (i = 0; iars[i][0] != 0 && i < MAX_IARS; i++) {
+ if (iars[i][0] == iar) {
+ iars[i][1]++;
+ i = MAX_IARS;
+ break;
+ }
+ }
+
+ if (i < MAX_IARS) {
+ iars[i][0] = iar;
+ iars[i][1] = 1;
+ }
+}
+
+extern const char *symbols_lookup(unsigned long addr,
+ unsigned long *symbolsize,
+ unsigned long *offset,
+ char *namebuf);
+
+if (smp_processor_id() == 0) {
+ int i;
+ long now = mftb();
+ long dump_interval = 3 * timebase_freq;
+
+ if (now > last_dump_time + dump_interval) {
+ last_dump_time = now;
+
+ for (i = 0; iars[i][0] != 0 && i < MAX_IARS; i++) {
+ char buff[128];
+
+ if (symbols_lookup(iars[i][0], NULL, NULL, buff))
+ printk("%4d %22s: %ld\n", i, buff, iars[i][1]);
+ else
+ printk("%4d %22lx: %ld\n", i, iars[i][0], iars[i][1]);
+ }
+ }
+}
+#endif
+
raise_softirq(TIMER_SOFTIRQ);
}
diff -r c94df1e4e62c xen/arch/powerpc/external.c
--- a/xen/arch/powerpc/external.c Thu Sep 14 15:43:38 2006 -0500
+++ b/xen/arch/powerpc/external.c Thu Sep 14 18:33:35 2006 -0400
@@ -80,6 +80,26 @@ void do_external(struct cpu_user_regs *r
BUG_ON(!(regs->msr & MSR_EE));
BUG_ON(mfmsr() & MSR_EE);
+#ifdef TRACE_DOMAINS
+if (smp_processor_id() == cpu_to_trace) {
+ int i;
+ long iar = regs->pc;
+
+ for (i = 0; iars[i][0] != 0 && i < MAX_IARS; i++) {
+ if (iars[i][0] == iar) {
+ iars[i][1]++;
+ i = MAX_IARS;
+ break;
+ }
+ }
+
+ if (i < MAX_IARS) {
+ iars[i][0] = iar;
+ iars[i][1] = 1;
+ }
+}
+#endif
+
vec = xen_mpic_get_irq(regs);
if (vec != -1) {
diff -r c94df1e4e62c xen/arch/powerpc/hcalls.c
--- a/xen/arch/powerpc/hcalls.c Thu Sep 14 15:43:38 2006 -0500
+++ b/xen/arch/powerpc/hcalls.c Thu Sep 14 18:33:35 2006 -0400
@@ -96,6 +96,26 @@ void do_hcall(struct cpu_user_regs *regs
{
ulong num = regs->gprs[3];
+#ifdef TRACE_DOMAINS
+if (smp_processor_id() == cpu_to_trace) {
+ int i;
+ long iar = regs->pc;
+
+ for (i = 0; iars[i][0] != 0 && i < MAX_IARS; i++) {
+ if (iars[i][0] == iar) {
+ iars[i][1]++;
+ i = MAX_IARS;
+ break;
+ }
+ }
+
+ if (i < MAX_IARS) {
+ iars[i][0] = iar;
+ iars[i][1] = 1;
+ }
+}
+#endif
+
local_irq_enable();
if ((num & XEN_MARK(0)) == XEN_MARK(0)) {
diff -r c94df1e4e62c xen/include/asm-powerpc/config.h
--- a/xen/include/asm-powerpc/config.h Thu Sep 14 15:43:38 2006 -0500
+++ b/xen/include/asm-powerpc/config.h Thu Sep 14 18:33:35 2006 -0400
@@ -38,6 +38,17 @@ extern char _end[];
extern char _end[];
extern char _etext[];
extern char __bss_start[];
+
+#de
[XenPPC] Assertion 'entry->next->prev == entry' failed
With current linux.hg and xen.hg, I am getting this consistently when creating my third domU: (XEN) Domain[2].0: initializing (XEN) Assertion 'entry->next->prev == entry' failed, line 92, file xen/include/xen/list.h (XEN) BUG at xen/include/xen/list.h:92 ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] JS20 hangs while 'Quiescing Open Firmware ...'
On Wed, Sep 13, 2006 at 11:58:07AM -0500, [EMAIL PROTECTED] wrote: > > This can be deceiving. Blades have 2 UARTs and SLOF will multiplex > > output to both, but input from one. > > JS20 blades have only one UART wired up. We actually soldered a connecter for the second serial port for some our JS20's. Hence the support for console=comX that went into Xen/PPC. But in general you are right. For a stock JS20 he should use console=com1 or no console argument at all. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Simple instruction trace facility
This very simple patch prints sampled statistics every three seconds
about what has been executing on a particular physical processor.
Example output for cpu 3 on a JS21 is:
(XEN)044efbc: 166
(XEN)142a810: 150090839
(XEN)2 c04ea660: 1
(XEN)3 c0037ba0: 1
For reference, 42a810 is in Xen's idle loop, and 44efbc is in it's sleep
implementation.
I don't think it is for submission, hopefully we will soon have a proper
high-performance trace facility accessed via an hcall and logic in the
dom0 kernel that allows a userspace consumer to drain it. I have
another version of this patch with per-cpu buffers and so on, but I'd
rather we focus on a real tracing facility. Is anybody working on this?
I am posting it here in case it is useful to other people, and because I
will soon post some statistics for domUs on secondary processors
gathered with it.
---
exceptions.c | 44
1 file changed, 44 insertions(+)
diff -r 82ddfe69c62e xen/arch/powerpc/exceptions.c
--- a/xen/arch/powerpc/exceptions.c Tue Sep 12 06:48:32 2006 -0400
+++ b/xen/arch/powerpc/exceptions.c Wed Sep 13 17:50:43 2006 -0400
@@ -29,6 +29,15 @@
#undef DEBUG
+#define TRACE_DOMAINS
+
+#ifdef TRACE_DOMAINS
+#define MAX_IARS 1000
+static long iars[MAX_IARS][2];
+static long last_dump_time = 0;
+static int cpu_to_trace = 3;
+#endif
+
extern ulong ppc_do_softirq(ulong orig_msr);
extern void do_timer(struct cpu_user_regs *regs);
extern void do_dec(struct cpu_user_regs *regs);
@@ -41,6 +50,41 @@ void do_timer(struct cpu_user_regs *regs
/* Set HDEC high so it stops firing and can be reprogrammed by
* set_preempt() */
mthdec(INT_MAX);
+
+#ifdef TRACE_DOMAINS
+if (smp_processor_id() == cpu_to_trace) {
+ int i;
+ long iar = regs->pc;
+
+ for (i = 0; iars[i][0] != 0 && i < MAX_IARS; i++) {
+ if (iars[i][0] == iar) {
+ iars[i][1]++;
+ i = MAX_IARS;
+ break;
+ }
+ }
+
+ if (i < MAX_IARS) {
+ iars[i][0] = iar;
+ iars[i][1] = 1;
+ }
+}
+
+if (smp_processor_id() == 0) {
+ int i;
+ long now = mftb();
+ long dump_interval = 3 * timebase_freq;
+
+ if (now > last_dump_time + dump_interval) {
+ last_dump_time = now;
+
+ for (i = 0; iars[i][0] != 0 && i < MAX_IARS; i++) {
+ printk("%4d %17lx: %ld\n", i, iars[i][0], iars[i][1]);
+ }
+ }
+}
+#endif
+
raise_softirq(TIMER_SOFTIRQ);
}
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix race in timebase sync logic
The SMP timebase sync code that recently went in has a race in which secondary processor X+1 can potentially take the timebase offset that is still in flight for secondary processor X. The cause of this was marking a secondary processor online from the boot cpu and then trying to wait for the secondary processor to mark itself online. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- setup.c |1 - 1 file changed, 1 deletion(-) diff -r 715014b95488 xen/arch/powerpc/setup.c --- a/xen/arch/powerpc/setup.c Thu Sep 07 22:09:06 2006 -0400 +++ b/xen/arch/powerpc/setup.c Fri Sep 08 11:26:27 2006 -0400 @@ -244,9 +244,8 @@ static int kick_secondary_cpus(int maxcp if (cpuid >= maxcpus) break; init_parea(cpuid); -cpu_set(cpuid, cpu_online_map); smp_generic_give_timebase(); /* wait for it */ while (!cpu_online(cpuid)) cpu_relax(); ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] fix build break in exceptions.c
Building exceptions.c with CRASH_DEBUG=y produces a build break, because
gcc complains about the unused variable.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
exceptions.c |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff -r 715014b95488 xen/arch/powerpc/exceptions.c
--- a/xen/arch/powerpc/exceptions.c Thu Sep 07 22:09:06 2006 -0400
+++ b/xen/arch/powerpc/exceptions.c Fri Sep 08 11:04:55 2006 -0400
@@ -55,11 +55,11 @@ void do_dec(struct cpu_user_regs *regs)
void program_exception(struct cpu_user_regs *regs, unsigned long cookie)
{
-int recover = 0;
-
#ifdef CRASH_DEBUG
__trap_to_gdb(regs, cookie);
#else /* CRASH_DEBUG */
+int recover = 0;
+
show_registers(regs);
printk("dar 0x%016lx, dsisr 0x%08x\n", mfdar(), mfdsisr());
printk("hid4 0x%016lx\n", regs->hid4);
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] padd_to_maddr panic
Using current xen.hg and linux.hg, I get this when I try to run `xm vcpu-list': (XEN) (XEN) Panic on CPU 0: (XEN) paddr_to_maddr: Dom:0 paddr: 0xf5d5ecc0 bad type:0x3 (XEN) (XEN) (XEN) Reboot in five seconds... Note that `xm list' works fine. The 0xf5d5ecc0 address is consistent across boots. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Overflow in decrementer restore
Just to provide background for this commit that went in today:
--- a/xen/arch/powerpc/powerpc64/domain.c
+++ b/xen/arch/powerpc/powerpc64/domain.c
@@ -55,7 +55,10 @@ void load_sprs(struct vcpu *v)
/* adjust the DEC value to account for cycles while not
* running this OS */
timebase_delta = mftb() - v->arch.timebase;
-v->arch.dec -= timebase_delta;
+if (timebase_delta > v->arch.dec)
+v->arch.dec = 0;
+else
+v->arch.dec -= timebase_delta;
}
In the patch titled "Schedule idle domain on secondary processors",
I mentioned that sometimes the entire system would freeze, so I didn't
want the patch to be considered for merging.
The problem turned out to be that we don't sync the timebases between
the processors. So if load_sprs() is executed on a different CPU than
save_sprs() was, the call to mftb is bogus. The timebase_delta can
overflow into a large unsigned value of up to 149 seconds on JS21. So
the domU was not wrecking the machine, the decrementer was just being
loaded with a huge value every time that domU's vcpu was loaded on a
particular physical CPU, including cpu0.
This patch also went in, to pin dom0 to cpu0:
--- a/xen/arch/powerpc/setup.c Fri Sep 01 12:31:56 2006 -0400
+++ b/xen/arch/powerpc/setup.c Fri Sep 01 12:37:29 2006 -0400
@@ -343,6 +343,10 @@ static void __init __start_xen(multiboot
if (NULL == alloc_vcpu(dom0, 0, 0))
panic("Error creating domain 0 vcpu 0\n");
+/* The Interrupt Controller will route everything to CPU 0 so we
+ * need to make sure Dom0's vVCPU 0 is pinned to the CPU */
+dom0->vcpu[0]->cpu_affinity = cpumask_of_cpu(0);
+
We are currently thinking about how best to sync the timebases. Right
now it looks like pulling in Linux's implementation is the best option.
Any comments would be appreciated.
We did have a real memory controller hang, as discussed on this list in
response to my original post. It only occurred on Maple, where PIBS
does not clear the HIOR for secondary CPUSs, so their first exeception was
delivered to 0xX00 + Y. Hence this patch that went in yesterday:
+cpu0_hior = 0;
+mthior(cpu0_hior);
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Fix paddr_to_maddr panic message
I saw this fire once and was mightily confused by the "type" printed.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
usercopy.c |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff -r e12aa1195f58 xen/arch/powerpc/usercopy.c
--- a/xen/arch/powerpc/usercopy.c Wed Aug 30 18:35:20 2006 -0400
+++ b/xen/arch/powerpc/usercopy.c Wed Aug 30 20:08:35 2006 -0400
@@ -52,8 +52,8 @@ static unsigned long paddr_to_maddr(unsi
case PFN_TYPE_LOGICAL:
break;
default:
-panic("%s: called with bad memory address type: 0x%lx\n",
-__func__, paddr);
+panic("%s: address 0x%lx has bad address type: 0x%x\n",
+__func__, paddr, mtype);
break;
}
pa <<= PAGE_SHIFT;
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] [PATCH/RFC] Schedule idle domain on secondary processors
On Tue, Aug 29, 2006 at 09:02:58AM +0200, Segher Boessenkool wrote: > >It is quite stable in that the secondary processors reliably join the > >idle domain and wait for free pages to scrub, handling 0x980 > >interrupts > >with no problem. > > What's this 980 exception? Perhaps my phrasing is bad. I was referring to the hypervisor decrementor interrupt (hdec). > >However, the domU's sometimes hang during initialization. When the > >domU hangs, it seems the whole machine freezes, including the serial > >console. > > Most common cause of this is hanging the U3/U4. Do you have a > hardware debugger to see where this happens? I had a friend take a look at the state of cpu 0, but everything seems ok. It looks like there is a race and occasionally one of the secondary processors is hanging the U4. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH/RFC] Schedule idle domain on secondary processors
This patch fixes memory corruption caused by start_of_day, and makes the
secondary processors join the idle domain and become eligible for domU
scheduling.
It is quite stable in that the secondary processors reliably join the
idle domain and wait for free pages to scrub, handling 0x980 interrupts
with no problem. I have been able to use `xm create' to launch Linux
domU's up to their bash prompt on the secondary processors.
However, the domU's sometimes hang during initialization. When the domU
hangs, it seems the whole machine freezes, including the serial console.
>From the description of the network-backed filesystem race with network
interface bringup, I don't think the serial console was affected. I am
usually able to create one or two domU's before the hang happens, but
sometimes it happens on the first attempt.
I'd appreciate any testing of this patch on Maple or JS20 hardware
and/or comments on what might be causing the machine to hang.
---
setup.c | 41 -
1 file changed, 20 insertions(+), 21 deletions(-)
diff -r f05a3e9d3e8f xen/arch/powerpc/setup.c
--- a/xen/arch/powerpc/setup.c Mon Aug 28 18:35:29 2006 -0500
+++ b/xen/arch/powerpc/setup.c Mon Aug 28 23:43:19 2006 -0400
@@ -88,6 +88,8 @@ extern void initialize_keytable(void);
volatile struct processor_area * volatile global_cpu_table[NR_CPUS];
+static struct domain *idle_domain;
+
int is_kernel_text(unsigned long addr)
{
if (addr >= (unsigned long) &_start &&
@@ -164,8 +166,6 @@ static void percpu_free_unused_areas(voi
static void __init start_of_day(void)
{
-struct domain *idle_domain;
-
init_IRQ();
scheduler_init();
@@ -180,23 +180,6 @@ static void __init start_of_day(void)
/* for some reason we need to set our own bit in the thread map */
cpu_set(0, cpu_sibling_map[0]);
-percpu_free_unused_areas();
-
-{
-/* FIXME: Xen assumes that an online CPU is a schedualable
- * CPU, but we just are not there yet. Remove this fragment when
- * scheduling processors actually works. */
-int cpuid;
-
-printk("WARNING!: Taking all secondary CPUs offline\n");
-
-for_each_online_cpu(cpuid) {
-if (cpuid == 0)
-continue;
-cpu_clear(cpuid, cpu_online_map);
-}
-}
-
initialize_keytable();
/* Register another key that will allow for the the Harware Probe
* to be contacted, this works with RiscWatch probes and should
@@ -253,8 +236,9 @@ static int kick_secondary_cpus(int maxcp
continue;
if (cpuid >= maxcpus)
break;
+
+cpu_set(cpuid, cpu_online_map);
init_parea(cpuid);
-cpu_set(cpuid, cpu_online_map);
}
return 0;
@@ -264,7 +248,19 @@ int secondary_cpu_init(int cpuid, unsign
int secondary_cpu_init(int cpuid, unsigned long r4);
int secondary_cpu_init(int cpuid, unsigned long r4)
{
+struct vcpu *vcpu;
+
cpu_initialize(cpuid);
+
+vcpu = alloc_vcpu(idle_domain, cpuid, cpuid);
+if (vcpu == NULL)
+BUG();
+
+set_current(idle_domain->vcpu[cpuid]);
+idle_vcpu[cpuid] = current;
+
+startup_cpu_idle_loop();
+
while(1);
}
@@ -337,6 +333,8 @@ static void __init __start_xen(multiboot
debugger_trap_immediate();
#endif
+start_of_day();
+
/* Deal with secondary processors. */
if (opt_nosmp) {
printk("nosmp: leaving secondary processors spinning forever\n");
@@ -345,7 +343,8 @@ static void __init __start_xen(multiboot
kick_secondary_cpus(max_cpus);
}
-start_of_day();
+/* This cannot be called before secondary cpus are marked online. */
+percpu_free_unused_areas();
/* Create initial domain 0. */
dom0 = domain_create(0);
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] domU bad page state
On Mon, Aug 28, 2006 at 09:23:17PM -0500, Hollis Blanchard wrote: > On Mon, 2006-08-28 at 22:13 -0400, Amos Waterland wrote: > Hmm, I'm not sure what could cause this. > > How reproducible is it? Does it only happen once per domU boot? Always > the same address/backtrace? Generally speaking, how old were the builds > you were using before (one day old? one week old?)? Every time, on a JS21 and JS20. Only happens once per domU boot, but happens for every domU boot. Same address and backtrace every time. I'm really suprised you are not seeing it. Make sure to scroll up in the console output of your domU boot ... ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] domU bad page state
Using tip of tree xen.hg and linux.hg, I get this when launching a domU: Freeing initrd memory: 3141k freed Bad page state in process 'swapper' page:c075d000 flags:0x1008 mapping: mapcount :0 count:0 Trying to fix it up, but a reboot is needed Backtrace: Call Trace: [C081BBA0] [C000F844] .show_stack+0x54/0x1f0 (unreliable) [C081BC50] [C0094B60] .bad_page+0x70/0xd0 [C081BCD0] [C009574C] .free_hot_cold_page+0x1fc/0x210 [C081BD70] [C00332BC] .free_initrd_mem+0x10c/0x150 [C081BE10] [C04F08F0] .free_initrd+0x70/0x130 [C081BEB0] [C00092D8] .init+0x168/0x430 [C081BF90] [C0028F54] .kernel_thread+0x4c/0x68 The domU completes its boot fine and I am able to run programs etc. I'm using the dom1 config that Jimi published earlier. ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Enable autofs in defconfig
Turn on autofs support in the Xen PPC defconfig. This allows dom0 to do automounts and is necessary for a portable nfsroot. Signed-off-by: Amos Waterland <[EMAIL PROTECTED]> --- xen_maple_defconfig |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff -r 899b84c2c979 arch/powerpc/configs/xen_maple_defconfig --- a/arch/powerpc/configs/xen_maple_defconfig Wed Aug 23 13:19:37 2006 -0500 +++ b/arch/powerpc/configs/xen_maple_defconfig Fri Aug 25 15:10:06 2006 -0400 @@ -1,7 +1,7 @@ # # Automatically generated make config: don't edit # Linux kernel version: 2.6.17 -# Tue Aug 8 09:35:08 2006 +# Fri Aug 25 15:08:08 2006 # CONFIG_PPC64=y CONFIG_64BIT=y @@ -1108,7 +1108,7 @@ CONFIG_INOTIFY_USER=y CONFIG_INOTIFY_USER=y # CONFIG_QUOTA is not set CONFIG_DNOTIFY=y -# CONFIG_AUTOFS_FS is not set +CONFIG_AUTOFS_FS=y # CONFIG_AUTOFS4_FS is not set # CONFIG_FUSE_FS is not set ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
Re: [XenPPC] Error creating domain on JS20 (Fw: [Prose-jvm] Brief Status in TRL (2006/08/24))
On Thu, Aug 24, 2006 at 03:19:58PM -0500, Hollis Blanchard wrote: > Here is the more general solution: it searches /cpus for the first node > with device_type containing "cpu", regardless of how the node is named. > > I won't commit until I hear a couple "works for me" reports (and no > "fails miserably" reports :) . I had the same problem on JS21, and Hollis' patch fixes it for me. Acked-by: Amos Waterland <[EMAIL PROTECTED]> ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Enable SMP, smp_processor_id, for_each_cpu, nosmp, maxcpus=X
Add support for the "nosmp" and "maxcpus=X" command line options, and
address Hollis' concerns about comments, prototypes, and panic messages.
Handshake with secondary processors, move all of them out of their OF
spinloop even if there are more of them than NR_CPUS, enumerate them
according to a logical rather than firmware numerical space, give them a
stack, make them run C code to initialize their HIDs, initialize their
r13 registers to a per-processor pointer, enable smp_processor_id and
and properly set the maps of present and online CPUs.
Tested on 2-way JS20, 4-way JS21, and 16-way systemsim-gpul, with
NR_CPUS from 1 to 64, nosmp, and maxcpus from 1 to 3.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/boot_of.c | 67 +++--
arch/powerpc/powerpc64/exceptions.S | 37
arch/powerpc/powerpc64/ppc970.c | 22 +
arch/powerpc/setup.c | 71 ++-
include/asm-powerpc/config.h |2
include/asm-powerpc/current.h|2
include/asm-powerpc/powerpc64/procarea.h |1
include/asm-powerpc/processor.h |2
include/asm-powerpc/smp.h|4 -
9 files changed, 188 insertions(+), 20 deletions(-)
diff -r 326e6736d92b xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cMon Aug 21 10:04:37 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cMon Aug 21 15:54:13 2006 -0400
@@ -31,6 +31,9 @@
#include
#include "exceptions.h"
#include "of-devtree.h"
+
+/* Secondary processors use this for handshaking with main processor. */
+volatile unsigned int __spin_ack;
static ulong of_vec;
static ulong of_msr;
@@ -955,7 +958,7 @@ static int __init boot_of_cpus(void)
static int __init boot_of_cpus(void)
{
int cpus;
-int cpu;
+int cpu, bootcpu, logical;
int result;
u32 cpu_clock[2];
@@ -980,10 +983,65 @@ static int __init boot_of_cpus(void)
cpu_khz /= 1000;
of_printf("OF: clock-frequency = %ld KHz\n", cpu_khz);
-/* FIXME: should not depend on the boot CPU bring the first child */
+/* Look up which CPU we are running on right now. */
+result = of_getprop(bof_chosen, "cpu", &bootcpu, sizeof (bootcpu));
+if (result == OF_FAILURE)
+of_panic("Failed to look up boot cpu\n");
+
cpu = of_getpeer(cpu);
-while (cpu > 0) {
-of_start_cpu(cpu, (ulong)spin_start, 0);
+
+/* We want a continuous logical cpu number space. */
+cpu_set(0, cpu_present_map);
+cpu_set(0, cpu_online_map);
+
+/* Spin up all CPUS, even if there are more than NR_CPUS, because
+ * Open Firmware has them spinning on cache lines which will
+ * eventually be scrubbed, which could lead to random CPU activation.
+ */
+for (logical = 1; cpu > 0; logical++) {
+unsigned int cpuid, ping, pong;
+unsigned long now, then, timeout;
+
+if (cpu == bootcpu) {
+of_printf("skipping boot cpu!\n");
+continue;
+}
+
+result = of_getprop(cpu, "reg", &cpuid, sizeof(cpuid));
+if (result == OF_FAILURE)
+of_panic("cpuid lookup failed\n");
+
+of_printf("spinning up secondary processor #%d: ", logical);
+
+__spin_ack = ~0x0;
+ping = __spin_ack;
+pong = __spin_ack;
+of_printf("ping = 0x%x: ", ping);
+
+mb();
+result = of_start_cpu(cpu, (ulong)spin_start, logical);
+if (result == OF_FAILURE)
+of_panic("start cpu failed\n");
+
+/* We will give the secondary processor five seconds to reply. */
+then = mftb();
+timeout = then + (5 * timebase_freq);
+
+do {
+now = mftb();
+if (now >= timeout) {
+of_printf("BROKEN: ");
+break;
+}
+
+mb();
+pong = __spin_ack;
+} while (pong == ping);
+of_printf("pong = 0x%x\n", pong);
+
+if (pong != ping)
+cpu_set(logical, cpu_present_map);
+
cpu = of_getpeer(cpu);
}
return 1;
@@ -1031,6 +1089,7 @@ multiboot_info_t __init *boot_of_init(
boot_of_rtas();
/* end of OF */
+of_printf("Quiescing Open Firmware ...\n");
of_call("quiesce", 0, 0, NULL);
return &mbi;
diff -r 326e6736d92b xen/arch/powerpc/powerpc64/exceptions.S
--- a/xen/arch/powerpc/powerpc64/exceptions.S Mon Aug 21 10:04:37 2006 -0400
+++ b/xen/arch/powerpc/powerpc64/exceptions.S Mon Aug 21 14:58:34 2006 -0400
@@ -514,6 +514,43 @@ _GLOBAL(sleep)
mtmsrd r3
blr
+/* The primary processor issues a firmware call to spin us up at this
+ * address, passing our CPU number in r3. We only need a function
[XenPPC] [PATCH] Enable smp_processor_id and present/online cpu maps
Handshake with secondary processors, move all of them out of their OF
spinloop even if there are more of them than NR_CPUS, give them a stack,
make them run C code to initialize their HIDs, initialize their r13
registers to a per-processor pointer, enable smp_processor_id and and
properly set the maps of present and online CPUs.
Note that this patch needs the "Linker script causes SMP memory corruption"
patch to work properly on large SMP systems. It supercedes
"Init secondary processors up to assigning r13 (respin)". Several
subtle MP issues are fixed in this patch versus previous versions.
Tested on 2-way Maple, 2-way JS20, 4-way JS21, and 16-way systemsim-gpul,
with NR_CPUS from 1 to 64.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/boot_of.c | 67 +--
arch/powerpc/powerpc64/exceptions.S | 36
arch/powerpc/powerpc64/ppc970.c | 22 +-
arch/powerpc/setup.c | 56 +
include/asm-powerpc/config.h |2
include/asm-powerpc/current.h|2
include/asm-powerpc/powerpc64/procarea.h |1
include/asm-powerpc/processor.h |2
include/asm-powerpc/smp.h|4 -
9 files changed, 172 insertions(+), 20 deletions(-)
diff -r 539a1e666982 xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cFri Aug 18 14:07:50 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cFri Aug 18 21:58:14 2006 -0400
@@ -31,6 +31,9 @@
#include
#include "exceptions.h"
#include "of-devtree.h"
+
+/* Secondary processors use this for handshaking with main processor. */
+volatile unsigned int __spin_ack;
static ulong of_vec;
static ulong of_msr;
@@ -955,7 +958,7 @@ static int __init boot_of_cpus(void)
static int __init boot_of_cpus(void)
{
int cpus;
-int cpu;
+int cpu, bootcpu, logical;
int result;
u32 cpu_clock[2];
@@ -980,10 +983,65 @@ static int __init boot_of_cpus(void)
cpu_khz /= 1000;
of_printf("OF: clock-frequency = %ld KHz\n", cpu_khz);
-/* FIXME: should not depend on the boot CPU bring the first child */
+/* Look up which CPU we are running on right now. */
+result = of_getprop(bof_chosen, "cpu", &bootcpu, sizeof (bootcpu));
+if (result == OF_FAILURE)
+of_panic("Failed to look up boot cpu\n");
+
cpu = of_getpeer(cpu);
-while (cpu > 0) {
-of_start_cpu(cpu, (ulong)spin_start, 0);
+
+/* We want a continuous logical cpu number space. */
+cpu_set(0, cpu_present_map);
+cpu_set(0, cpu_online_map);
+
+/* Spin up all CPUS, even if there are more than NR_CPUS, because
+ * Open Firmware has them spinning on cache lines which will
+ * eventually be scrubbed, which could lead to random CPU activation.
+ */
+for (logical = 1; cpu > 0; logical++) {
+unsigned int cpuid, ping, pong;
+unsigned long now, then, timeout;
+
+if (cpu == bootcpu) {
+of_printf("skipping boot cpu!\n");
+continue;
+}
+
+result = of_getprop(cpu, "reg", &cpuid, sizeof(cpuid));
+if (result == OF_FAILURE)
+of_panic("cpuid lookup failed\n");
+
+of_printf("spinning up secondary processor #%d: ", logical);
+
+__spin_ack = ~0x0;
+ping = __spin_ack;
+pong = __spin_ack;
+of_printf("ping = 0x%x: ", ping);
+
+mb();
+result = of_start_cpu(cpu, (ulong)spin_start, logical);
+if (result == OF_FAILURE)
+of_panic("start cpu failed\n");
+
+/* We will give the secondary processor five seconds to reply. */
+then = mftb();
+timeout = then + (5 * timebase_freq);
+
+do {
+now = mftb();
+if (now >= timeout) {
+of_printf("BROKEN: ");
+break;
+}
+
+mb();
+pong = __spin_ack;
+} while (pong == ping);
+of_printf("pong = 0x%x\n", pong);
+
+if (pong != ping)
+cpu_set(logical, cpu_present_map);
+
cpu = of_getpeer(cpu);
}
return 1;
@@ -1031,6 +1089,7 @@ multiboot_info_t __init *boot_of_init(
boot_of_rtas();
/* end of OF */
+of_printf("Quiescing Open Firmware ...\n");
of_call("quiesce", 0, 0, NULL);
return &mbi;
diff -r 539a1e666982 xen/arch/powerpc/powerpc64/exceptions.S
--- a/xen/arch/powerpc/powerpc64/exceptions.S Fri Aug 18 14:07:50 2006 -0400
+++ b/xen/arch/powerpc/powerpc64/exceptions.S Fri Aug 18 21:06:13 2006 -0400
@@ -514,6 +514,42 @@ _GLOBAL(sleep)
mtmsrd r3
blr
+/* The primary processor issues a firmware call to spin us up at this
+ * address, passing our CPU number in r
[XenPPC] [PATCH] Linker script causes SMP memory corruption
To understand the context for this patch, observe that the following is
the case in stock Xen/PPC:
$ nm xen-syms | egrep ' per_cpu__|__per_cpu' | sort
0044c680 d per_cpu__domain_shuttingdown
0044c688 D per_cpu__mc_state
0044c700 d per_cpu__t_timer
...
0045cc80 D per_cpu__tlbflush_time
0045d2b0 A __per_cpu_data_end
0045d2b0 A __per_cpu_start
0046 A __per_cpu_end
So __per_cpu_start and __per_cpu_data_end are identical, and none of the
per_cpu variable are actually between __per_cpu_start and __per_cpu_end.
When CPU #20 on a Xen/PPC SMP system tries to load or store to
its t_timer per-cpu variable, a calculation like this will take place:
address = 0x44c700 + (20 << 12)
So the address is 0x460700, which beyond __per_cpu_end among other
problems. For my configuration, it lies in the __hypercall_table, so
when init_timer(per_cpu(t_timer)) is run, a portion of the hypercall
table is memset to zero and then some timer-related function pointers
are written to it. Many cycles later, when init_hcalls is run, a
garbage pointer is dereferenced and a 0x200 exception occurs.
Also, the linker script has a bogus assumption that the per-cpu
section size can be calculated by adding (NR_CPUS << PERCPU_SHIFT) to
__per_cpu_start. It needs to use __per_cpu_data_end instead, or a
different instance of memory corruption can occur.
Also, a .data.percpu section was not actually being created in the final
linked object.
This patch fixes the three issues described above. Tested on
systemsim-gpul, JS20 and JS21.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/xen.lds.S | 15 +--
include/asm-powerpc/percpu.h |5 +++--
2 files changed, 12 insertions(+), 8 deletions(-)
diff -r 539a1e666982 xen/arch/powerpc/xen.lds.S
--- a/xen/arch/powerpc/xen.lds.SFri Aug 18 14:07:50 2006 -0400
+++ b/xen/arch/powerpc/xen.lds.SSat Aug 19 01:53:55 2006 -0400
@@ -124,12 +124,15 @@ SECTIONS
.inithcall.text : { *(.inithcall.text) }
__inithcall_end = .;
- __per_cpu_start = .;
- .data.percpu : { *(.data.percpu) }
- __per_cpu_data_end = .;
- . = __per_cpu_start + (NR_CPUS << PERCPU_SHIFT);
- . = ALIGN(STACK_SIZE);
- __per_cpu_end = .;
+ .data.percpu : {
+__per_cpu_start = .;
+*(__per_cpu)
+__per_cpu_data_end = .;
+. = __per_cpu_data_end + (NR_CPUS << PERCPU_SHIFT);
+__per_cpu_multiple_end = .;
+. = ALIGN(STACK_SIZE);
+__per_cpu_end = .;
+ }
/* end Xen addition */
diff -r 539a1e666982 xen/include/asm-powerpc/percpu.h
--- a/xen/include/asm-powerpc/percpu.h Fri Aug 18 14:07:50 2006 -0400
+++ b/xen/include/asm-powerpc/percpu.h Sat Aug 19 01:07:21 2006 -0400
@@ -8,8 +8,9 @@
/* Separate out the type, so (int[3], foo) works. */
#define DEFINE_PER_CPU(type, name) \
-__attribute__((__section__(".data.percpu")))\
-__typeof__(type) per_cpu__##name
+__typeof__(type) per_cpu__##name \
+__attribute__((section("__per_cpu")))
+
/* var is in discarded region: offset to particular copy we want */
#define per_cpu(var, cpu) \
___
Xen-ppc-devel mailing list
Xen-ppc-devel@lists.xensource.com
http://lists.xensource.com/xen-ppc-devel
[XenPPC] [PATCH] Init secondary processors up to assigning r13 (respin)
Respin because the -Wshadow portion went in today.
Re-tested on systemsim-gpul with up to four simulated processors,
Maple-D with two processors, JS20 with two processors, and JS21 with
four processors.
This patch brings the secondary processors up to the point where they
have a per-processor structure pointer in r13, at which point they just
enter a spinloop. If you test it, you should see two additional pieces
of output on your console:
spinning up secondary processor #1: ping = 0x: pong = 0x1
spinning up secondary processor #2: ping = 0x: pong = 0x2
spinning up secondary processor #3: ping = 0x: pong = 0x3
(XEN) CPU #1: Hello World! SP = 1fe60 TOC = 757e78 HID0 = 5111818000
(XEN) CPU #2: Hello World! SP = 1be60 TOC = 757e78 HID0 = 5111818000
(XEN) CPU #3: Hello World! SP = 79fe60 TOC = 757e78 HID0 = 5111818000
I think this patch is a candidate for merging. I will submit a patch
next for smp_processor_id, and then one for making the secondary
processors join the idle domain, but I would like to have any necessary
discussion about this portion first.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/boot_of.c | 53 +---
arch/powerpc/powerpc64/exceptions.S | 31 +
arch/powerpc/powerpc64/ppc970.c | 22 --
arch/powerpc/setup.c| 52 ++-
include/asm-powerpc/config.h|2 -
include/asm-powerpc/current.h |2 -
include/asm-powerpc/processor.h |2 -
7 files changed, 147 insertions(+), 17 deletions(-)
diff -r 539a1e666982 xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cFri Aug 18 14:07:50 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cFri Aug 18 14:55:34 2006 -0400
@@ -31,6 +31,9 @@
#include
#include "exceptions.h"
#include "of-devtree.h"
+
+/* Secondary processors use this for handshaking with main processor. */
+volatile unsigned int __spin_ack;
static ulong of_vec;
static ulong of_msr;
@@ -955,7 +958,7 @@ static int __init boot_of_cpus(void)
static int __init boot_of_cpus(void)
{
int cpus;
-int cpu;
+int cpu, bootcpu;
int result;
u32 cpu_clock[2];
@@ -980,10 +983,53 @@ static int __init boot_of_cpus(void)
cpu_khz /= 1000;
of_printf("OF: clock-frequency = %ld KHz\n", cpu_khz);
-/* FIXME: should not depend on the boot CPU bring the first child */
+/* Look up which CPU we are running on right now. */
+result = of_getprop(bof_chosen, "cpu", &bootcpu, sizeof (bootcpu));
+if (result == OF_FAILURE)
+of_panic("Failed to look up boot cpu\n");
+
cpu = of_getpeer(cpu);
while (cpu > 0) {
-of_start_cpu(cpu, (ulong)spin_start, 0);
+unsigned int cpuid, ping, pong;
+unsigned long now, then, timeout;
+
+if (cpu == bootcpu) {
+of_printf("skipping boot cpu!\n");
+continue;
+}
+
+result = of_getprop(cpu, "reg", &cpuid, sizeof(cpuid));
+if (result == OF_FAILURE)
+of_panic("cpuid lookup failed\n");
+
+of_printf("spinning up secondary processor #%d: ", cpuid);
+
+__spin_ack = ~0x0;
+ping = __spin_ack;
+pong = __spin_ack;
+of_printf("ping = 0x%x: ", ping);
+
+mb();
+result = of_start_cpu(cpu, (ulong)spin_start, cpuid);
+if (result == OF_FAILURE)
+of_panic("start cpu failed\n");
+
+/* We will give the secondary processor five seconds to reply. */
+then = mftb();
+timeout = then + (5 * timebase_freq);
+
+do {
+now = mftb();
+if (now >= timeout) {
+of_printf("BROKEN: ");
+break;
+}
+
+mb();
+pong = __spin_ack;
+} while (pong == ping);
+of_printf("pong = 0x%x\n", pong);
+
cpu = of_getpeer(cpu);
}
return 1;
@@ -1031,6 +1077,7 @@ multiboot_info_t __init *boot_of_init(
boot_of_rtas();
/* end of OF */
+of_printf("Quiescing Open Firmware ...\n");
of_call("quiesce", 0, 0, NULL);
return &mbi;
diff -r 539a1e666982 xen/arch/powerpc/powerpc64/exceptions.S
--- a/xen/arch/powerpc/powerpc64/exceptions.S Fri Aug 18 14:07:50 2006 -0400
+++ b/xen/arch/powerpc/powerpc64/exceptions.S Fri Aug 18 04:21:12 2006 -0400
@@ -514,6 +514,37 @@ _GLOBAL(sleep)
mtmsrd r3
blr
+/* The primary processor issues a firmware call to spin us up at this
+ * address, passing our CPU number in r3. We only need a function
+ * entry point instead of a descriptor since this is never called from
+ * C code.
+ */
.globl spin_start
spin_start:
+/* Write our processor number as an acknowledgme
[XenPPC] [PATCH] Init secondary processors up to assigning r13 (take two)
This patch brings the secondary processors up to the point where they
have a per-processor structure pointer in r13, at which point they just
enter a spinloop. It differs from take one in that it has an important
shadowed variable bug fix. If you test it, you should see two
additional pieces of output on your console:
spinning up secondary processor #1: ping = 0x: pong = 0x1
spinning up secondary processor #2: ping = 0x: pong = 0x2
spinning up secondary processor #3: ping = 0x: pong = 0x3
(XEN) CPU #1: Hello World! SP = 1fe60 TOC = 757e78 HID0 = 5111818000
(XEN) CPU #2: Hello World! SP = 1be60 TOC = 757e78 HID0 = 5111818000
(XEN) CPU #3: Hello World! SP = 79fe60 TOC = 757e78 HID0 = 5111818000
I think this patch is a candidate for merging. I will submit a patch
next for smp_processor_id, and then one for making the secondary
processors join the idle domain, but I would like to have any necessary
discussion about this portion first.
Note that this patch also makes arch/powerpc build with -Wshadow, since
otherwise it is way too easy to accidentaly shadow the global parea
variable which is supposed to only be referred to in r13.
Tested on systemsim-gpul with up to four simulated processors, Maple-D
with two processors, JS20 with two processors, and JS21 with four processors.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/Makefile |2 -
arch/powerpc/boot_of.c | 53 +---
arch/powerpc/mpic.c |2 -
arch/powerpc/ofd_fixup.c|4 +-
arch/powerpc/powerpc64/exceptions.S | 31 +
arch/powerpc/powerpc64/ppc970.c | 22 --
arch/powerpc/setup.c| 52 ++-
include/asm-powerpc/config.h|2 -
include/asm-powerpc/current.h |2 -
include/asm-powerpc/processor.h |2 -
10 files changed, 151 insertions(+), 21 deletions(-)
diff -r 7855f2e80748 xen/arch/powerpc/Makefile
--- a/xen/arch/powerpc/Makefile Thu Aug 17 20:38:08 2006 -0400
+++ b/xen/arch/powerpc/Makefile Thu Aug 17 22:44:27 2006 -0400
@@ -46,7 +46,7 @@ obj-y += elf32.o
# These are extra warnings like for the arch/ppc directory but may not
# allow the rest of the tree to build.
-PPC_C_WARNINGS += -Wundef -Wmissing-prototypes -Wmissing-declarations
+PPC_C_WARNINGS += -Wundef -Wmissing-prototypes -Wmissing-declarations -Wshadow
CFLAGS += $(PPC_C_WARNINGS)
LINK=0x40
diff -r 7855f2e80748 xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cThu Aug 17 20:38:08 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cThu Aug 17 22:44:27 2006 -0400
@@ -31,6 +31,9 @@
#include
#include "exceptions.h"
#include "of-devtree.h"
+
+/* Secondary processors use this for handshaking with main processor. */
+volatile unsigned int __spin_ack;
static ulong of_vec;
static ulong of_msr;
@@ -955,7 +958,7 @@ static int __init boot_of_cpus(void)
static int __init boot_of_cpus(void)
{
int cpus;
-int cpu;
+int cpu, bootcpu;
int result;
u32 cpu_clock[2];
@@ -980,10 +983,53 @@ static int __init boot_of_cpus(void)
cpu_khz /= 1000;
of_printf("OF: clock-frequency = %ld KHz\n", cpu_khz);
-/* FIXME: should not depend on the boot CPU bring the first child */
+/* Look up which CPU we are running on right now. */
+result = of_getprop(bof_chosen, "cpu", &bootcpu, sizeof (bootcpu));
+if (result == OF_FAILURE)
+of_panic("Failed to look up boot cpu\n");
+
cpu = of_getpeer(cpu);
while (cpu > 0) {
-of_start_cpu(cpu, (ulong)spin_start, 0);
+unsigned int cpuid, ping, pong;
+unsigned long now, then, timeout;
+
+if (cpu == bootcpu) {
+of_printf("skipping boot cpu!\n");
+continue;
+}
+
+result = of_getprop(cpu, "reg", &cpuid, sizeof(cpuid));
+if (result == OF_FAILURE)
+of_panic("cpuid lookup failed\n");
+
+of_printf("spinning up secondary processor #%d: ", cpuid);
+
+__spin_ack = ~0x0;
+ping = __spin_ack;
+pong = __spin_ack;
+of_printf("ping = 0x%x: ", ping);
+
+mb();
+result = of_start_cpu(cpu, (ulong)spin_start, cpuid);
+if (result == OF_FAILURE)
+of_panic("start cpu failed\n");
+
+/* We will give the secondary processor five seconds to reply. */
+then = mftb();
+timeout = then + (5 * timebase_freq);
+
+do {
+now = mftb();
+if (now >= timeout) {
+of_printf("BROKEN: ");
+break;
+}
+
+mb();
+pong = __spin_ack;
+} while (pong == ping);
+of_printf("pong = 0x%x\n", pong);
+
[XenPPC] [PATCH] Init secondary processors up to assigning r13
This patch brings the secondary processors up to the point where they
have a per-processor structure pointer in r13, at which point they just
enter a spinloop. If you test it, you should see two additional pieces
of output on your console:
spinning up secondary processor #1: ping = 0x: pong = 0x1
spinning up secondary processor #2: ping = 0x: pong = 0x2
spinning up secondary processor #3: ping = 0x: pong = 0x3
(XEN) CPU #1: Hello World! SP = 1fe60 TOC = 757e78 HID0 = 5111818000
(XEN) CPU #2: Hello World! SP = 1be60 TOC = 757e78 HID0 = 5111818000
(XEN) CPU #3: Hello World! SP = 79fe60 TOC = 757e78 HID0 = 5111818000
I think this patch is a candidate for merging. I will submit a patch
next for smp_processor_id, and then one for making the secondary
processors join the idle domain, but I would like to have any necessary
discussion about this portion first.
Note that this patch also makes arch/powerpc build with -Wshadow, since
otherwise it is way too easy to accidentaly shadow the global parea
variable which is supposed to only be referred to in r13.
Tested on systemsim-gpul with up to four simulated processors, Maple-D
with two processors, JS20 with two processors, and JS21 with four processors.
Signed-off-by: Amos Waterland <[EMAIL PROTECTED]>
---
arch/powerpc/Makefile |2 -
arch/powerpc/boot_of.c | 53 +---
arch/powerpc/mpic.c |2 -
arch/powerpc/ofd_fixup.c|4 +-
arch/powerpc/powerpc64/exceptions.S | 31 +
arch/powerpc/powerpc64/ppc970.c | 22 --
arch/powerpc/setup.c| 52 ++-
include/asm-powerpc/config.h|2 -
include/asm-powerpc/current.h |3 +-
include/asm-powerpc/processor.h |2 -
10 files changed, 152 insertions(+), 21 deletions(-)
diff -r 7855f2e80748 xen/arch/powerpc/Makefile
--- a/xen/arch/powerpc/Makefile Thu Aug 17 20:38:08 2006 -0400
+++ b/xen/arch/powerpc/Makefile Thu Aug 17 22:24:30 2006 -0400
@@ -46,7 +46,7 @@ obj-y += elf32.o
# These are extra warnings like for the arch/ppc directory but may not
# allow the rest of the tree to build.
-PPC_C_WARNINGS += -Wundef -Wmissing-prototypes -Wmissing-declarations
+PPC_C_WARNINGS += -Wundef -Wmissing-prototypes -Wmissing-declarations -Wshadow
CFLAGS += $(PPC_C_WARNINGS)
LINK=0x40
diff -r 7855f2e80748 xen/arch/powerpc/boot_of.c
--- a/xen/arch/powerpc/boot_of.cThu Aug 17 20:38:08 2006 -0400
+++ b/xen/arch/powerpc/boot_of.cThu Aug 17 22:24:30 2006 -0400
@@ -31,6 +31,9 @@
#include
#include "exceptions.h"
#include "of-devtree.h"
+
+/* Secondary processors use this for handshaking with main processor. */
+volatile unsigned int __spin_ack;
static ulong of_vec;
static ulong of_msr;
@@ -955,7 +958,7 @@ static int __init boot_of_cpus(void)
static int __init boot_of_cpus(void)
{
int cpus;
-int cpu;
+int cpu, bootcpu;
int result;
u32 cpu_clock[2];
@@ -980,10 +983,53 @@ static int __init boot_of_cpus(void)
cpu_khz /= 1000;
of_printf("OF: clock-frequency = %ld KHz\n", cpu_khz);
-/* FIXME: should not depend on the boot CPU bring the first child */
+/* Look up which CPU we are running on right now. */
+result = of_getprop(bof_chosen, "cpu", &bootcpu, sizeof (bootcpu));
+if (result == OF_FAILURE)
+of_panic("Failed to look up boot cpu\n");
+
cpu = of_getpeer(cpu);
while (cpu > 0) {
-of_start_cpu(cpu, (ulong)spin_start, 0);
+unsigned int cpuid, ping, pong;
+unsigned long now, then, timeout;
+
+if (cpu == bootcpu) {
+of_printf("skipping boot cpu!\n");
+continue;
+}
+
+result = of_getprop(cpu, "reg", &cpuid, sizeof(cpuid));
+if (result == OF_FAILURE)
+of_panic("cpuid lookup failed\n");
+
+of_printf("spinning up secondary processor #%d: ", cpuid);
+
+__spin_ack = ~0x0;
+ping = __spin_ack;
+pong = __spin_ack;
+of_printf("ping = 0x%x: ", ping);
+
+mb();
+result = of_start_cpu(cpu, (ulong)spin_start, cpuid);
+if (result == OF_FAILURE)
+of_panic("start cpu failed\n");
+
+/* We will give the secondary processor five seconds to reply. */
+then = mftb();
+timeout = then + (5 * timebase_freq);
+
+do {
+now = mftb();
+if (now >= timeout) {
+of_printf("BROKEN: ");
+break;
+}
+
+mb();
+pong = __spin_ack;
+} while (pong == ping);
+of_printf("pong = 0x%x\n", pong);
+
cpu = of_getpeer(cpu);
}
return 1;
@@ -1031,6 +1077,7 @@ multiboot_info_t
