Re: [Xen-ia64-devel] One unstablity in fast syscall path
Le Mardi 13 Juin 2006 02:36, Tian, Kevin a écrit : Recently we kept seeing intermittent domU creation failure after creating VTI domain. After some debug, we found it caused by following changeset: changeset: 10238:b27139d8c1e1 user:[EMAIL PROTECTED] date:Sat Jun 03 11:16:47 2006 -0600 summary: [IA64] paravirtualize vdso There're several privileged accesses to vPSR in gate page, and previously it's done directly by triggering GP fault without xenlinux changes. Rev 10238 tries to para-virtualize those instructions by introducing two new hyperprivops. Now a simple assumption is forced to distinguish break immediate by checking vPSR.ic. People thought there's no break instruction with psr.ic off in xenlinux. So any break with vpsr.ic off is considered as a hyperprivop between domain and hypervisor. Based on this assumption, explicit vpsr.ic clearance is required before hypercall, and then recover required after. OK, that works, for most time because normally those hypercalls are issued within kernel text segment which is mapped by TR. So after hypervisor finished service for fast hypercall, xen can always RFI to exact resume point in xenlinux even when ITLB miss happens since that resume point can be found by vTR. However, gate page is not mapped by TR, and thus populating PSR.ic at non-TR-mapped segment is even mad in native environment. In our observation, when backing to gate page after servicing fast hypercall, ITLB miss happens but there's no guest entry in vTLB and VHPT. Thus xen injects a nested dtlb fault to xenlinux (vPSR.ic is still off) which is undesired to crash domU. That's the real problem, though we're not sure why this phenomenon is easier to be reproduced after creating VTI domain. Quick/easy solution can be to roll back above changeset to ensure tree stability first, and then community needs to think more robust solution later. Good work ! After reading some Xen/ia64 source, I think we'd better not to use vpsr.ic for fast hypercall: it has some interractions with vpsr.ic flag! I'd vote for creating a new paravirtualized register: xen_break (or xen_hyperprivop): when this new register is set, breaks are hyperprivop, when not set breaks are reflected. Of course, changing the logic requires some work and careful checks. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] how to make a smp xen0 config file, thanks
Hi Tristan, I reboot my machine with your configuration. Cset is 10022, when it booted up I find only one cpu in the /proc/cpuinfo. But xm vcpu-list showed me: NameID VCPU CPU State Time(s) CPU Affinity Domain-0 0 00 r-- 317.9 0 Domain-0 0 1- --p 0.0 any cpu Very strange it is! Why they are different? Thank you zhangjingke -Original Message- From: Tristan Gingold [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 3:43 PM To: Zhang, Jingke Cc: xen-ia64-devel@lists.xensource.com Subject: Re: [Xen-ia64-devel] how to make a smp xen0 config file, thanks Le Mardi 13 Juin 2006 09:05, Zhang, Jingke a écrit : Hi, Would you please tell me how to write the config file to boot up a smp xen0? I wrote: append=com2=57600,8n1 console=com2 sched=bvt dom0_mem=1024M -- nomca maxcpus=2 console=tty0 console=ttyS1,57600,8n1 root=/dev/sda2 but when my machine started up, I only found one cpu. Yes, the default is 1 cpu. You should add dom0_max_vcpus=2 in xen part of command line. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] how to make a smp xen0 config file, thanks
Le Mardi 13 Juin 2006 10:02, Zhang, Jingke a écrit : Hi Tristan, I reboot my machine with your configuration. Cset is 10022, when it booted up I find only one cpu in the /proc/cpuinfo. But xm vcpu-list showed me: Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 00 r-- 317.9 0 Domain-0 0 1- --p 0.0 any cpu Very strange it is! Why they are different? Linux didn't boot the second cpu. Could you post the linux console messages ? Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] how to make a smp xen0 config file, thanks
Using Cset 10269, I was also able to boot dom0 with 2 vcpus. But the /proc/cpuinfo only have one cpu while xm vcpu-lis has two. Here is the serial console info (Cset 10269): Xen version 3.0-unstable (build@) (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2) ) Sat Jun 10 01:02:38 CST 2006 Latest ChangeSet: Fri Jun 9 10:40:31 2006 -0600 10269:aa2298739112 (XEN) xen image pstart: 0x400, xenheap pend: 0x800 (XEN) efi.trim_top: ignoring 4KB of memory at 0x0 due to granule hole at 0x0 (XEN) efi.trim_top: ignoring 24KB of memory at 0x1000 due to granule hole at 0x0 (XEN) efi.trim_top: ignoring 8KB of memory at 0x7000 due to granule hole at 0x0 (XEN) efi.trim_top: ignoring 484KB of memory at 0x9000 due to granule hole at 0x 0 (XEN) efi.trim_top: ignoring 4KB of memory at 0x84000 due to granule hole at 0x0 (XEN) efi.trim_top: ignoring 108KB of memory at 0x85000 due to granule hole at 0 x0 (XEN) efi.trim_bottom: ignoring 15360KB of memory at 0x10 due to granule hol e at 0x0 (XEN) ready to move Dom0 to 0x800 with len e78a90...ready to move initrd to 0x8e7c000 with len 13e84d...Done (XEN) find_memory: efi_memmap_walk returns max_page=bffed (XEN) Before heap_start: f411a7f0 (XEN) After heap_start: f4138000 (XEN) Init boot pages: 0x100 - 0x400. (XEN) Init boot pages: 0x8e78a90 - 0x8e7c000. (XEN) Init boot pages: 0x8fba84d - 0x7f708000. (XEN) Init boot pages: 0x7fe58000 - 0x7ff3. (XEN) Init boot pages: 0x1 - 0x1c000. (XEN) Init boot pages: 0x28000 - 0x2fee04010. (XEN) Init boot pages: 0x2fee04070 - 0x2fee07f66. (XEN) Init boot pages: 0x2fee07fd1 - 0x2fef4e010. (XEN) Init boot pages: 0x2fef4ea00 - 0x2ffe14000. (XEN) Init boot pages: 0x2ffe8 - 0x2fffb4000. (XEN) System RAM: 8167MB (8363152kB) (XEN) size of virtual frame_table: 20480kB (XEN) virtual machine to physical table: f3a00098 size: 4144kB (XEN) max_page: 0xbffed (XEN) Xen heap: 62MB (64288kB) (XEN) About to call scheduler_init() (XEN) Using scheduler: Borrowed Virtual Time (bvt) (XEN) ACPI: RSDP (v002 INTEL ) @ 0x7ff83 000 (XEN) ACPI: XSDT (v001 INTEL SR870BN4 0x01072002 MSFT 0x00010013) @ 0x7 ff83090 (XEN) ACPI: FADT (v003 INTEL SR870BN4 0x01072002 MSFT 0x00010013) @ 0x7 ff83138 (XEN) ACPI: MADT (v001 INTEL SR870BN4 0x01072002 MSFT 0x00010013) @ 0x7 ff83230 (XEN) ACPI: DSDT (v001 Intel SR870BN4 0x INTL 0x20030918) @ 0x0 000 (XEN) SAL 3.20: Intel Corp SR870BN4 version 3.0 (XEN) SAL Platform features: BusLock (XEN) SAL: AP wakeup using external interrupt vector 0xf0 (XEN) cpu package is Multi-Core capable: number of cores=2 (XEN) cpu package is Multi-Threading capable: number of siblings=2 (XEN) avail:0x31700740, status:0x740,control:0x3170, vm?0x100 (XEN) WARNING: no opcode provided from hardware(0)!!! (XEN) vm buffer size: 1048576, order: 6 (XEN) cpu_init: current=f40cc000 (XEN) vm_buffer: 0xf7e0 (XEN) vhpt_init: vhpt paddr=0x1fffe, end=0x1fffe (XEN) iosapic_system_init: Disabling PC-AT compatible 8259 interrupts (XEN) ACPI: Local APIC address e800fee0 (XEN) ACPI: LSAPIC (acpi_id[0x00] lsapic_id[0xc0] lsapic_eid[0x18] enabled) (XEN) CPU 0 (0xc018) enabled (BSP) (XEN) ACPI: LSAPIC (acpi_id[0x01] lsapic_id[0xc2] lsapic_eid[0x18] disabled) (XEN) CPU 1 (0xc218) disabled (XEN) ACPI: LSAPIC (acpi_id[0x02] lsapic_id[0xc4] lsapic_eid[0x18] disabled) (XEN) CPU 2 (0xc418) disabled (XEN) ACPI: LSAPIC (acpi_id[0x03] lsapic_id[0xc6] lsapic_eid[0x18] disabled) (XEN) CPU 3 (0xc618) disabled (XEN) ACPI: LSAPIC (acpi_id[0x04] lsapic_id[0xc1] lsapic_eid[0x18] enabled) (XEN) CPU 4 (0xc118) enabled (XEN) ACPI: LSAPIC (acpi_id[0x05] lsapic_id[0xc3] lsapic_eid[0x18] disabled) (XEN) CPU 5 (0xc318) disabled (XEN) ACPI: LSAPIC (acpi_id[0x06] lsapic_id[0xc5] lsapic_eid[0x18] disabled) (XEN) CPU 6 (0xc518) disabled (XEN) ACPI: LSAPIC (acpi_id[0x07] lsapic_id[0xc7] lsapic_eid[0x18] disabled) (XEN) CPU 7 (0xc718) disabled (XEN) ACPI: LSAPIC (acpi_id[0x08] lsapic_id[0xc0] lsapic_eid[0x98] disabled) (XEN) CPU 8 (0xc098) disabled (XEN) ACPI: LSAPIC (acpi_id[0x09] lsapic_id[0xc2] lsapic_eid[0x98] disabled) (XEN) CPU 9 (0xc298) disabled (XEN) ACPI: LSAPIC (acpi_id[0x0a] lsapic_id[0xc4] lsapic_eid[0x98] disabled) (XEN) CPU 10 (0xc498) disabled (XEN) ACPI: LSAPIC (acpi_id[0x0b] lsapic_id[0xc6] lsapic_eid[0x98] disabled) (XEN) CPU 11 (0xc698) disabled (XEN) ACPI: LSAPIC (acpi_id[0x0c] lsapic_id[0xc1] lsapic_eid[0x98] disabled) (XEN) CPU 12 (0xc198) disabled (XEN) ACPI: LSAPIC (acpi_id[0x0d] lsapic_id[0xc3] lsapic_eid[0x98] disabled) (XEN) CPU 13 (0xc398) disabled (XEN) ACPI: LSAPIC (acpi_id[0x0e] lsapic_id[0xc5] lsapic_eid[0x98] disabled) (XEN) CPU 14 (0xc598) disabled (XEN) ACPI: LSAPIC (acpi_id[0x0f] lsapic_id[0xc7] lsapic_eid[0x98] disabled)
Re: [Xen-ia64-devel] how to make a smp xen0 config file, thanks
Le Mardi 13 Juin 2006 10:41, Zhang, Jingke a écrit : Using Cset 10269, I was also able to boot dom0 with 2 vcpus. But the /proc/cpuinfo only have one cpu while xm vcpu-lis has two. Here is the serial console info (Cset 10269): Only cpus 0 and 4 are enabled. Xen+linux may not like this setup! Could you enable more cpus (at least cpu #1) ? What does happen if you boot linux (without Xen) ? Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] how to make a smp xen0 config file, thanks
Hi Tristan, That machine disabled HT from BIOS. Xen should also only can see 2, why 4? Native Linux hasn't problem run on it. Best Regards, Yongkang (Kangkang) 永康 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tristan Gingold Sent: 2006年6月13日 16:59 To: Zhang, Jingke Cc: xen-ia64-devel@lists.xensource.com Subject: Re: [Xen-ia64-devel] how to make a smp xen0 config file, thanks Le Mardi 13 Juin 2006 10:41, Zhang, Jingke a écrit : Using Cset 10269, I was also able to boot dom0 with 2 vcpus. But the /proc/cpuinfo only have one cpu while xm vcpu-lis has two. Here is the serial console info (Cset 10269): Only cpus 0 and 4 are enabled. Xen+linux may not like this setup! Could you enable more cpus (at least cpu #1) ? What does happen if you boot linux (without Xen) ? Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Re: [Xen-devel] [patch] architecture-specific ELF header checking
On 12 Jun 2006, at 21:12, Hollis Blanchard wrote: This patch has only been compile-tested on x86, but it should be pretty straightforward. It could break IA64 since it adds checks they weren't doing before, but I would expect their ELF binaries are labeled properly. I am not keen on adding loads of -D CFLAGS options for very localised/specific macros. They could go in a per-arch header file, but I think in this case just having ifdef's in xc_elf.h is clean enough. Nothing outside the ELF-parsing code should be looking at these values so keeping them private is sensible. Apart from that, the general idea is fine so I'll modify and apply. -- Keir ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] how to make a smp xen0 config file, thanks
Le Mardi 13 Juin 2006 10:58, You, Yongkang a écrit : Hi Tristan, That machine disabled HT from BIOS. Xen should also only can see 2, why 4? Native Linux hasn't problem run on it. Ok. You have only one socket, two cores enabled. Right ? I still do not understand why Linux doesn't start the second CPU. Xen seems to be correct. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] how to make a smp xen0 config file, thanks
From: Tristan Gingold Sent: 2006年6月13日 17:43 Le Mardi 13 Juin 2006 10:58, You, Yongkang a écrit : Hi Tristan, That machine disabled HT from BIOS. Xen should also only can see 2, why 4? Native Linux hasn't problem run on it. Ok. You have only one socket, two cores enabled. Right ? I still do not understand why Linux doesn't start the second CPU. Xen seems to be correct. Tristan. The hint should be: (XEN) ACPI: Local APIC address c000fee0 (XEN) ACPI: [APIC:0x07] ignored 12 entries of 16 found Currently the default config file sets CONFIG_NR_CPUS to be 4, which means only the early 4 entries can be parsed. See Jingke's log: (XEN) ACPI: LSAPIC (acpi_id[0x00] lsapic_id[0xc0] lsapic_eid[0x18] enabled) (XEN) CPU 0 (0xc018) enabled (BSP) ... (XEN) ACPI: LSAPIC (acpi_id[0x04] lsapic_id[0xc1] lsapic_eid[0x18] enabled) (XEN) CPU 4 (0xc118) enabled The 2nd active LSAPIC entry happens to be 5th slot in MADT table, which is ignored by dom0. Then it's easy to understand why Tristan can succeed on his box due to possibly different LSAPIC entries. Jingke, could you try enlarging CONFIG_NR_CPUS to be same as MAX_VIRT_CPUS, to see whether dom0 smp will be up for you? Thanks, Kevin ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] One unstablity in fast syscall path
On Mon, Jun 12, 2006 at 09:49:17PM -0600, Alex Williamson wrote: On Tue, 2006-06-13 at 08:36 +0800, Tian, Kevin wrote: That's the real problem, though we're not sure why this phenomenon is easier to be reproduced after creating VTI domain. Quick/easy solution can be to roll back above changeset to ensure tree stability first, and then community needs to think more robust solution later. Hi Kevin, Thanks for tracking this down, I've seen some instability lately too. Isaku, should we back this one out until it can be stabilized or do you have a more direct solution? Thanks, Sorry for the regression. Please back it out. The correctness has higher priority than performance, the patch should be backed out since it focuses on only performance. There are the following choices for right fix, I think. A. use hypercall instead of hyperprivop. B. introduce new flag for hyperprivop instread VPSR.ic as Tristan proposed. C. use one of itr to map vDSO. D. abondan vDSO paravirtualization. Thanks. -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] One unstablity in fast syscall path
From: Isaku Yamahata [mailto:[EMAIL PROTECTED] Sent: 2006年6月13日 19:13 There are the following choices for right fix, I think. A. use hypercall instead of hyperprivop. B. introduce new flag for hyperprivop instread VPSR.ic as Tristan proposed. C. use one of itr to map vDSO. D. abondan vDSO paravirtualization. Option B seems reasonable since original vPSR.ic is also borrowed to serve as an indicator. Thanks, Kevin ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] kernel BUG at mm/page_alloc.c:1116!
Hello. Presumably the memory descriptors which Xen/IA64 creates for the domain doesn't cover xenLinux kernel text area. If so, it is a bug of Xen/IA64. Which domain are you trying to create, dom0 or domU? (maybe domU. just to make sure) Thanks. On Mon, Jun 12, 2006 at 10:10:42AM -0400, Rodrigo Lord wrote: Hi! I`m testing the xen/ia64 and the error below appears when I tryed to boot a virtual machine with 80 MB of memory... kernel BUG at mm/page_alloc.c:1116! swapper[1]: bugcheck! 0 [1] Modules linked in: Pid: 1, CPU 0, comm: swapper psr : 0010085a6010 ifs : 8288 ip : [a001000dec50] Not tainted ip is at free_pages+0xb0/0x100 unat: pfs : 8288 rsc : 000b rnat: e3736d30 bsps: e12b0480 pr : 5a41 ldrs: ccv : fpsr: 0009804c8a70433f csd : ssd : b0 : a001000dec50 b6 : b7 : a001004eb5c0 f6 : 0fffbc8c0 f7 : 0ffd9a200 f8 : 08000 f9 : 10002a000 f10 : 0fffbc8c0 f11 : 1003e r1 : a00100d63310 r2 : 0001 r3 : a00100b810ec r8 : 0027 r9 : a00100b810ec r10 : 0001 r11 : f101 r12 : e12efe20 r13 : e12e8000 r14 : 0001 r15 : r16 : f1004c18 r17 : f1004c20 r18 : a00100b810ec r19 : a00100b810ec r20 : f101 r21 : 0001 r22 : f1004c18 r23 : 000d r24 : 0001 r25 : 07ff r26 : 0089 r27 : 4000 r28 : r29 : a00100b803b8 r30 : a00100b80398 r31 : a00100b77fb8 Call Trace: [a00100019260] show_stack+0x40/0xa0 sp=e12ef990 bsp=e12e9108 [a00100019af0] show_regs+0x7d0/0x800 sp=e12efb60 bsp=e12e90c0 [a0010003fac0] die+0x1c0/0x3e0 sp=e12efb60 bsp=e12e9078 [a0010003fd20] die_if_kernel+0x40/0x60 sp=e12efb80 bsp=e12e9048 [a0010003ff50] ia64_bad_break+0x210/0x460 sp=e12efb80 bsp=e12e9020 [a00100067720] xen_leave_kernel+0x0/0x3b0 sp=e12efc50 bsp=e12e9020 [a001000dec50] free_pages+0xb0/0x100 sp=e12efe20 bsp=e12e8fd8 [a001000628f0] free_initmem+0x150/0x200 sp=e12efe20 bsp=e12e8fb8 [a00100011130] init+0x230/0x400 sp=e12efe30 bsp=e12e8f98 [a0010001b430] kernel_thread_helper+0x30/0x60 sp=e12efe30 bsp=e12e8f70 [a00100010d40] start_kernel_thread+0x20/0x40 sp=e12efe30 bsp=e12e8f70 0Kernel panic - not syncing: Attempted to kill init! I tried to boot with 90MB, 128MB, and more... and it`s ok! Is there a minimal memory to set? Can It be a possible bug? Thanks. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] kernel BUG at mm/page_alloc.c:1116!
On Mon, 2006-06-12 at 10:10 -0400, Rodrigo Lord wrote: Hi! I`m testing the xen/ia64 and the error below appears when I tryed to boot a virtual machine with 80 MB of memory... kernel BUG at mm/page_alloc.c:1116! swapper[1]: bugcheck! 0 [1] Modules linked in: I think you'll find the same limitations with linux-ia64. IA64 systems aren't typically optimized for such low memory conditions. The per CPU area can take a significant chunk of memory, so make sure your NR_CPUS is configured to as little as possible. The kernel typically gets loaded at offset 68MB, making it difficult to go much below where you are. Thanks, Alex -- Alex Williamson HP Open Source Linux Org. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] RE: PATCH: merge iva
The reason that there are two groups of privileged registers, one in privregs (directly accessible by the guest) and one in arch_vcpu (not directly accesible) is that arch_vcpu is for registers that are not performance-sensitive AND might otherwise need to be validated before every use. For example, iva is used every time an interruption is reflected to a guest, which happens many thousands of times/second. If the guest could randomly (maliciously or accidentally) change iva, Xen should re-validate it before using it (e.g. to ensure that it is not in Xen address space, to ensure it is not an I/O address etc.) By allowing it to be changed only via the privileged instruction (trapped/emulated), it need only be validated when set (once at boot time for Linux). I realize validation may not be fully implemented (and there may be some registers in the wrong place), but that's the intent. -Original Message- From: Tristan Gingold [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 7:54 AM To: xen-ia64-devel@lists.xensource.com; Magenheimer, Dan (HP Labs Fort Collins); Williamson, Alex (Linux Kernel Dev) Subject: PATCH: merge iva Hi, this patch removes the iva field of arch_vcpu and use the iva field of privregs instead. Tested by booting dom0. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH] add volatile to pte entry of the p2m table
On Sat, 2006-06-10 at 11:50 +0900, Isaku Yamahata wrote: Hi Alex. On Fri, Jun 09, 2006 at 11:06:52AM -0600, Alex Williamson wrote: On Fri, 2006-06-09 at 18:16 +0900, Isaku Yamahata wrote: add volatile to pte entry of the p2m table. p2m table are shared by cpu. added volatile as compiler barrier. Are all of these really needed? Seems a little overkill to me. Actually no. assign_new_domain_page() and its families and domain_page_mapped() are used only when domain construction (for now). So their pte accesses aren't racy. However, to remove volatile of them, non-volatile version of lookup_alloc_domain_pte() is needed. Because domain creation isn't performance critical, I did't think it was worthwhile to both volatile version and non-volatile version of lookup_alloc_domain_pte(). Hi Isaku, I guess the problem is that I don't quite see how adding these volatile declarations actually fix anything. If we're dealing with races because of concurrent accesses to the data, I would expect we need to add some locking. I'd be surprised if declaring a variable as volatile would solve the race. Thanks, Alex -- Alex Williamson HP Open Source Linux Org. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [patch] [3/4] Support INIT handler of xen
On Mon, 2006-06-12 at 15:06 +0900, Akio Takebe wrote: fix macro to support INIT handler arch/ia64/linux-xen/minstate.h | 70 ++-- arch/ia64/linux-xen/unwind.c| 23 ++ include/asm-ia64/linux-xen/asm/system.h |5 +- 3 files changed, 67 insertions(+), 31 deletions(-) Hi Akio, There's quite a bit changed in this patch that doesn't seem to be covered by #ifdef XEN. Could you look into adding those in? Thanks, Alex -- Alex Williamson HP Open Source Linux Org. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Xen/ia64 OLS attendees?
Hi all, Is anyone going to the Ottawa Linux Symposium next month? I'll be there, curious if anyone else from the Xen/ia64 community was planning to attend. Thanks, Alex -- Alex Williamson HP Open Source Linux Org. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [patch] [4/4] Support INIT handler of xen
Hi Akio,
A few more comments...
On Mon, 2006-06-12 at 15:08 +0900, Akio Takebe wrote:
diff -r aa2298739112 xen/arch/ia64/linux-xen/Makefile
--- a/xen/arch/ia64/linux-xen/Makefile Fri Jun 09 10:40:31 2006 -0600
+++ b/xen/arch/ia64/linux-xen/Makefile Mon Jun 12 03:21:27 2006 +0900
@@ -3,6 +3,8 @@ obj-y += irq_ia64.o
obj-y += irq_ia64.o
obj-y += mm_contig.o
obj-y += pal.o
+obj-y += mca_asm.o
+obj-y += mca.o
Need a README.origin update too.
+#include linux/interrupt.h
+#include linux/irq.h
+#ifdef XEN
+#include xen/symbols.h
+#else
+#include linux/kallsyms.h
+#endif /* XEN */
+#include linux/smp_lock.h
+#include linux/bootmem.h
+#include linux/acpi.h
+#include linux/timer.h
+#include linux/module.h
+#include linux/kernel.h
+#include linux/smp.h
+#ifndef XEN
+#include linux/workqueue.h
+#endif /* !XEN */
I wouldn't bother with the /* XEN */ when there's so little code
between the #ifdef and #endif.
+#include asm/delay.h
+#include asm/machvec.h
+#include asm/page.h
+#include asm/ptrace.h
+#include asm/system.h
+#include asm/sal.h
+#include asm/mca.h
+
+#include asm/irq.h
+#include asm/hw_irq.h
+#ifndef XEN
+#include asm/crashdump.h
+#endif /* !XEN */
+#ifdef XEN
#else?
+#ifdef XEN
+fetch_min_state(ms, pt, sw);
+spin_lock(show_stack_lock);
+show_min_state(ms);
+printk(Backtrace of current vcpu (vcpu_id %d)\n,
current-vcpu_id);
+unw_init_from_interruption(info, current, pt, sw);
+ia64_do_show_stack(info, NULL);
+unw_init_running(save_ksp, NULL);
+spin_unlock(show_stack_lock);
+wmb();
+ init_cache_flush();
+
+if (spin_trylock(init_dump_lock)){
+#ifdef CONFIG_SMP
+udelay(5*100);
+#endif
+
+if(try_crashdump(pt) == 0)
+printk(\nINIT dump complete. Please reboot
now.\n);
+}
+ printk(%s: CPU%d init handler done\n,
__FUNCTION__ ,smp_processor_id());
I'm not completely sure of the leverage we're getting by wedging this
#ifdef XEN section into the existing init_handler_platform() function.
Maybe we should #ifdef out the whole function and make our own for xen?
In fact, the number of non-#ifdef'd out code in both mca.c and
mca_asm.S is fairly small. Are we planning to leverage more code out of
these in the future? I wonder if we'd be ahead to cut and paste the
little bits we are using into xen-specific files(?). Thanks,
Alex
--
Alex Williamson HP Open Source Linux Org.
___
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] One unstablity in fast syscall path
On Tue, Jun 13, 2006 at 01:11:04PM -0700, Magenheimer, Dan (HP Labs Fort Collins) wrote: There are two purposes of paravirtualization: one is correctness and the other is performance. If fully virtualized vDSO works properly and there's no impact on performance, it shouldn't be paravirtualized. If there is a measurable impact, it should be paravirtualized. I fully agree with you that paravirtualization for performance must be backed by a sort of measurement. I have a question on priv_handle_op(). I changed the function so that xen/ia64 reflects itlb miss to a domain when xen/ia64 fails to read a bundle. Xen/ia64 reflected dtlb miss before my change. Is it correct to reflect dtlb miss? I guess you already encountered the same problem and gave the consideration on it. With my change, the following sequence happens on every system call entry. It would causes performance loss. So I thought it was worthwhile to paravirtualize vDSO area. 1. a domain enters to vDSO 2. execute privilieged op. (rsm or reading psr) 3. traps to Xen/IA64 4. Xen/IA64 tries to read the operation, but fails because the vDSO page is only mapped by itlb cache. (This depends on processor tlb cache implementation) So it reflects itlb. 5. a domain issued itlb insert and xen/ia64 cached vcpu-arch.itlb 6. a domain re-execute the privileged op. 7. traps to Xen/IA64 8. Xen/IA64 successes to read the op with vcpu-arch.itlb and emulates the op. If I understand the problem (not sure I do fully), there is: E. use hyperprivops for vDSO but via a function call to code in hypercall.S (which *is* pinned). This options sounds good. Thanks. -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH] add volatile to pte entry of the p2m table
Hello Alex. On Tue, Jun 13, 2006 at 02:24:13PM -0600, Alex Williamson wrote: I guess the problem is that I don't quite see how adding these volatile declarations actually fix anything. If we're dealing with races because of concurrent accesses to the data, I would expect we need to add some locking. I'd be surprised if declaring a variable as volatile would solve the race. Thanks, I see. I'll write and post the documentation on SMP. -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] [RFC] SMP issues
Hello.
This is my memo on Xen/IA64 SMP. Please comment/request.
Taming races and memory ordering would become very subtle and error-prone.
It is very likely that I miss something. Please correct me.
NOTE: In this documentation, I discuss only on non-VT-i domain.
VT-i domain has the different scheme of vTLB. So another consideration
is needed.
The two patches are attached to help to under stand the discussion.
But they are not for commit.
* shared structures
There are some structures which are accessed by CPUs concurrently.
Here is the list of shared structures and operations on them which
read/write the structures.
- struct page_info
This is a xen global resource. This structure is accessed by
any CPUs.
operations on this structure:
- get_page() and its variant
- put_page() and its variant
- vTLB
vcpu-arch.{d, i}tlb: Software tlb cache. These are per VCPU data.
DEFINE_PER_CPU (unsigned long, vhpt_paddr): VHPT table per physical CPU.
domain_flush_vtlb_range() and domain_flush_vtlb_all()
write vcpu-arch.{d, i}tlb and VHPT table of vcpu which isn't current.
So there are potential races to read/write VHPT and vcpu-arch.{d, i}tlb.
Please note that reading VHPT is done by hardware page table walker.
operations on this structure:
- global tlb purge
vcpu_ptc_g(), vcpu_ptc_ga() and domain_page_flush()
I.e. callers of domain_flush_vtlb_range() and domain_flush_vtlb_all()
These functions invalidate VHPT entry and vcpu-arch.{i, d}tlb
- tlb insert
vcpu_itc_i()
vcpu_itc_d()
ia64_do_page_fault()
These functions set VHPT entry and vcpu-arch.{i, d}tlb.
Actually vcpu_itc_no_srlz() does.
- the P2M table
domain-mm and pgd, pud, pmd, pte table page.
This structure is used to convert domain pseudo physical address
to machine address. This is per domain resource.
operations on this structure:
- populate the P2M table tree
lookup_alloc_domain_pte() and its variants.
- set p2m entry
assign_new_domain_page() and its variants.
assign_domain_page() and its variants.
- xchg p2m entry
assign_domain_page_replace()
- cmpxchg p2m entry
assign_domain_page_cmpxchg_rel()
destroy_grant_host_mapping()
steal_page_for_grant_transfer()
zap_domain_page_one()
- read p2m entry
lookup_alloc_domain_pte() and its variants.
- the M2P table
mpt_table (or machine_to_phys_mapping)
This is a table which converts from machine address to pseudo physical
address. This is a global structure.
operations on this structure:
- set m2p entry
set_gpfn_from_mfn()
- zap m2p entry
set_gpfn_from_mfn(INVALID_P2M_ENTRY)
- get m2p entry
get_gpfn_from_mfn()
* avoiding races
The resources which are shared by CPUs must be accessed carefully
to avoid race.
IA64 has weak memory ordering so that attention must be paid
to access shared structures. [SDM vol2 PartII chap. 2]
- struct page_info memory ordering
get_page() has acquire semantics.
put_page() has release semantics.
- populating the p2m table
pgd, pud, pmd are append only.
- races when updating the P2M tables and the M2P table
The P2M entry are shared by more than one vcpu.
So they are accessed atomic operations.
I.e. xchg or cmpxchg must be used to update the p2m entry.
NOTE: When creating/destructing a domain, we don't need to take care of
this race.
The M2P table is inverse of the P2M table.
I.e. P2M(M2P(p)) = p and M2P(P2M(m)) = m
The M2P table and P2M table must be updated consistently.
Here is the update sequence
xchg or cmpxchg case
- set_gpfn_from_mfn(new_mfn, gpfn)
- memory barrier
- atomic update of the p2m entry (xchg or cmpxchg the p2m entry)
get old_mfn entry as a result.
- memory barrier
- set_gpfn_from_mfn(old_mfn, INVALID_P2M_ENTRY)
Here memory barrier can be achieved by release semantics.
- races between global tlb purge and tlb insert
This is a race between reading/writing vcpu-arch.{d, i}tlb or VHPT entry.
When a vcpu is about to insert tlb, another vcpu may purge tlb
cache globally. Inserting tlb (vcpu_itc_no_srlz()) or global tlb purge
(domain_flush_vtlb_range() and domain_flush_vtlb_all()) can't update
cpu-arch.{d, i}tlb, VHPT and mTLB. So there is a race here.
Use sequence lock to avoid this race.
After inserting tlb entry, check the sequence lock and retry to insert.
This means that when global tlb purge and tlb insert are issued
simultaneously, always tlb insert happens after global tlb purge.
There was an attempt to resolve this race by checking only
vcpu-arch.{d, i}tlb.p bit. However it was incomplete because it doesn't
take care of VHPT.
- races between p2m entry update and tlb insert
This is a race between reading/writing the p2m entry.
reader: vcpu_itc_i(), vcpu_itc_d(), ia64_do_page_fault()
writer: assign_domain_page_cmpxchg_rel(), destroy_grant_host_mapping(),
steal_page_for_grant_transfer(), zap_domain_page_one()
For example,
