Re: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609
Le Mercredi 27 Septembre 2006 12:05, You, Yongkang a écrit : The 2 XenU coexisting testing is easy to fail. I try 3 times to create 2 XenU domains at the same time (xm create config1 ) ; xm create config2 . The first 2 times are pass. But the 3rd time. XenU creating failed. After that, xm list can not list domains and will report Error: Device 768 not connected Use xend stop and xend start could recover xm list. But if try to create 2 XenU domains again, there will be 4 Zombie-Domains in xm list. It is same with the bug http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=747 Hi, Is it a recent bug ? According to the bugzilla date it seems so (Aug 29). Could you try to find the changeset from which this bug appears ? Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Re: blkbk/netbk modules fail to load on fedora xen/ia64
Hi, Aron and all netbk/xennet modules fail to load by using xen-ia64-unstable again. This issue is caused by the following patches. http://xenbits.xensource.com/ext/xen-ia64-unstable.hg?cs=c757ebffd500 We can probably resolve this issue. 1. Tristan's xencomm patches is applied. (I test by using xen-ia64-unstable, but these patches have some issue.) 2. revert the below patch. (I have not test yet.) 3. built in these modules. (I have not test yet.) I think the best way is 1. But Tristan's patches still have some issues. What do you think about this? Best Regards, Akio Takebe Akio Takebe wrote: [Thu Sep 21 2006, 09:53:16PM EDT] I can solve this problem by using dom0_mem=1G. This allows blkbk/netbk to load, but I guess a domU won't install yet. xenguest-install fails with a panic in dom0. I used this command: xenguest-install -n domu -r 1024 -f /root/domu.img \ -l http://hummer.zko.hp.com/fedora/linux/core/development/ia64/os/ \ -s 4 --nographics -p -x 'ide0=noprobe ide1=noprobe ide2=noprobe ide3= noprobe' Here is the output on the dom0 console: (XEN) ### domain f7bb4080: rid=8-c mp_rid=2000 (XEN) arch_domain_create: domain=f7bb4080 (XEN) DomainU EFI build up: ACPI 2.0=0x1000 (XEN) dom mem: type=13, attr=0x8008, range=[0x- 0x1000) (4KB) (XEN) dom mem: type=10, attr=0x8008, range=[0x1000- 0x2000) (4KB) (XEN) dom mem: type= 6, attr=0x8008, range=[0x2000- 0x3000) (4KB) (XEN) dom mem: type= 7, attr=0x0008, range=[0x3000- 0x3fff4000) (1023MB) (XEN) dom mem: type=12, attr=0x0001, range=[0x0c00- 0x1000) (64MB) audit(1158891786.948:4): dev=vif1.0 prom=256 old_prom=0 auid=4294967295 kernel unaligned access to 0xa002009e405f, ip=0xa00100295e91 kernel unaligned access to 0xa002009e4067, ip=0xa00100295e91 kernel unaligned access to 0xa002009e406f, ip=0xa00100295e91 kernel unaligned access to 0xa002009e4077, ip=0xa00100295e91 kernel unaligned access to 0xa002009e407f, ip=0xa00100295e91 (XEN) vcpu_get_lrr0: Unmasked interrupts unsupported (XEN) vcpu_get_lrr1: Unmasked interrupts unsupported (XEN) Linux version 2.6.18-1.2679.fc6xen ([EMAIL PROTECTED] com) (gcc version 4.1.1 20060917 (Red Hat 4.1.1-23)) #1 SMP Wed Sep 20 01: 18:10 EDT 2006 EFI v1.00 by Xen/ia64: SALsystab=0x2178 ACPI 2.0=0x1000 rsvd_region[0]: [0xe0002228, 0xe00022f0) rsvd_region[1]: [0xe0003000, 0xe0003030) rsvd_region[2]: [0xe400, 0xe4c2899b) rsvd_region[3]: [0xe4c2c000, 0xe597ac00) rsvd_region[4]: [0xe0003fff4000, 0xe0003fff8000) rsvd_region[5]: [0x, 0x) Initial ramdisk at: 0xe4c2c000 (13954048 bytes) SAL 0.1: Xen/ia64 Xen/ia64 version 0.0 SAL: AP wakeup using external interrupt vector 0xf3 xen_pal_emulator: UNIMPLEMENTED PAL CALL 42 (XEN) No logical to physical processor mapping available ACPI: Local APIC address c000fee0 ACPI: Error parsing MADT - no IOSAPIC entries 1 CPUs available, 1 CPUs total Running on Xen! start_info_pfn=0xfffd nr_pages=65536 flags=0x0 *** CALLED SAL_MC_SET_PARAMS. IGNORED... (XEN) *** CALLED SAL_MC_SET_PARAMS. IGNORED... (XEN) *** CALLED SAL_SET_VECTORS 0. IGNORED... (XEN) *** CALLED SAL_SET_VECTORS 1. IGNORED... (XEN) MCA related initialization done SMP: Allowing 1 CPUs, 0 hotplug CPUs Built 1 zonelists. Total pages: 61440 Kernel command line: method=http://hummer.zko.hp.com/fedora/linux/core/ development/ia64/os/ ide0=noprobe ide1=noprobe ide2=noprobe ide3=noprobe ide_setup: ide0=noprobe ide_setup: ide1=noprobe ide_setup: ide2=noprobe ide_setup: ide3=noprobe PID hash table entries: 4096 (order: 12, 32768 bytes) Console: colour dummy device 80x25 (file=grant_table.c, line=704) gnttab_transfer: error writing resp 0/1 kernel BUG at drivers/xen/netback/netback.c:631! swapper[0]: bugcheck! 0 [1] Modules linked in: loop xt_physdev bridge netloop netbk blkbk autofs4 hidp rfcomm l2cap bluetooth sunrpc ip_conntrack_netbios_ns ipt_REJECT iptable_filter ip_tables xt_state ip_conntrack nfnetlink xt_tcpudp ip6table_filter ip6_tables x_tables ipv6 vfat fat dm_multipath button parport_pc lp parport ide_cd sg e1000 cdrom dm_snapshot dm_zero dm_mirror dm_mod mptspi mptscsih mptbase scsi_transport_spi sd_mod scsi_mod ext3 jbd ehci_hcd ohci_hcd uhci_hcd Pid: 0, CPU 0, comm: swapper psr : 001008026010 ifs : 8694 ip : [a00200c80590] Not tainted ip is at net_rx_action+0x990/0x17a0 [netbk] unat: pfs : 8694 rsc : 0008 rnat: bsps: pr : 00019665 ldrs: ccv : fpsr: 0009804c8a70433f csd : ssd : b0 : a00200c80590 b6 : a001000b80c0 b7 :
RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11635
Hi Tristan, We found it for a long time. This issue also can be easily reproduced in latest changeset 11635. I guess it might be the common issue with IA32 side. But IA32 xenU didn't meet the problem with the same operations. Sorry for the wrong subject 11609, I send out yesterday. It should be 11635. Best Regards, Yongkang (Kangkang) 永康 -Original Message- From: Tristan Gingold [mailto:[EMAIL PROTECTED] Sent: 2006年9月28日 14:41 To: You, Yongkang; xen-ia64-devel@lists.xensource.com Subject: Re: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609 Le Mercredi 27 Septembre 2006 12:05, You, Yongkang a écrit : The 2 XenU coexisting testing is easy to fail. I try 3 times to create 2 XenU domains at the same time (xm create config1 ) ; xm create config2 . The first 2 times are pass. But the 3rd time. XenU creating failed. After that, xm list can not list domains and will report Error: Device 768 not connected Use xend stop and xend start could recover xm list. But if try to create 2 XenU domains again, there will be 4 Zombie-Domains in xm list. It is same with the bug http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=747 Hi, Is it a recent bug ? According to the bugzilla date it seems so (Aug 29). Could you try to find the changeset from which this bug appears ? Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] Re: ia64 kexec: xen - linux
Le Jeudi 28 Septembre 2006 03:27, Horms a écrit : On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote: Linux and xen call efi in real mode if set_virtual_address_map fails. You may add an option in both xen and linux to force calling efi in real mode. This should be really simple and you will be able to make progress. Great, I will test this out and see how it goes. The only possible drawback is performance. What kind of performance issues would you expect? Making EFI calls in physical mode is slower: Linux must switch from and to virtual mode. However EFI calls are very unfrequent so the impact should be almost nul. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH 1/12]MCA handler support for Xen/ia64 TAKE 2
Hi Alex, Thank you for your comment. I attached updated patches following to the comment. On Fri, 2006-09-22 at 19:32 +0900, SUZUKI Kazuhiro wrote: [1/12] patch for MCA handler.[mca-mca.patch] Looks good, a couple minor comments: * It looks like we're not returning a value for several functions that specify a return type. Please make sure the code compiles cleanly. I'm sorry but I cannot find such functions. Please teach me which functions correspond. It is confirmed that no warnings are found when compiling. * #define'ing mod_timer to set_timer may help remove #ifdef XEN in a few places. I defined `mod_timer' macro and removed several #ifdef XEN. * You might make a comment about the origin of disable_irq_nosync() and enable_irq() since they appear to be unchanged copies of the generic IRQ functions from Linux. I put the comment like Copy from linux/kernel/irq/manage.c. Thanks, KAZ Signed-off-by: Yutaka Ezaki [EMAIL PROTECTED] Signed-off-by: Masaki Kanno [EMAIL PROTECTED] Signed-off-by: Kazuhiro Suzuki [EMAIL PROTECTED] diff -r 3e4fa8b5b245 xen/arch/ia64/linux-xen/mca.c --- a/xen/arch/ia64/linux-xen/mca.c Tue Sep 12 11:43:22 2006 -0600 +++ b/xen/arch/ia64/linux-xen/mca.c Thu Sep 28 12:03:22 2006 +0900 @@ -80,6 +80,9 @@ #ifdef XEN #include xen/symbols.h #include xen/mm.h +#include xen/event.h +#include xen/softirq.h +#include asm/xenmca.h #endif #if defined(IA64_MCA_DEBUG_INFO) @@ -107,18 +110,26 @@ unsigned long __per_cpu_mca[NR_CPUS]; /* In mca_asm.S */ extern voidia64_monarch_init_handler (void); extern voidia64_slave_init_handler (void); +#ifdef XEN +extern void setup_vector (unsigned int vec, struct irqaction *action); +#endif static ia64_mc_info_t ia64_mc_info; -#ifndef XEN #define MAX_CPE_POLL_INTERVAL (15*60*HZ) /* 15 minutes */ #define MIN_CPE_POLL_INTERVAL (2*60*HZ) /* 2 minutes */ #define CMC_POLL_INTERVAL (1*60*HZ) /* 1 minute */ #define CPE_HISTORY_LENGTH5 #define CMC_HISTORY_LENGTH5 +#ifndefXEN static struct timer_list cpe_poll_timer; static struct timer_list cmc_poll_timer; +#else /* XEN */ +#definemod_timer(timer, expires) set_timer(timer, expires) +static struct timer cpe_poll_timer; +static struct timer cmc_poll_timer; +#endif /* * This variable tells whether we are currently in polling mode. * Start with this in the wrong state so we won't play w/ timers @@ -135,11 +146,9 @@ static int cpe_poll_enabled = 1; static int cpe_poll_enabled = 1; extern void salinfo_log_wakeup(int type, u8 *buffer, u64 size, int irqsafe); -#endif /* !XEN */ static int mca_init; -#ifndef XEN /* * IA64_MCA log support */ @@ -156,11 +165,24 @@ typedef struct ia64_state_log_s static ia64_state_log_t ia64_state_log[IA64_MAX_LOG_TYPES]; +#ifndefXEN #define IA64_LOG_ALLOCATE(it, size) \ {ia64_state_log[it].isl_log[IA64_LOG_CURR_INDEX(it)] = \ (ia64_err_rec_t *)alloc_bootmem(size); \ ia64_state_log[it].isl_log[IA64_LOG_NEXT_INDEX(it)] = \ (ia64_err_rec_t *)alloc_bootmem(size);} +#else /* XEN */ +#define IA64_LOG_ALLOCATE(it, size) \ + do { \ + unsigned int pageorder; \ + pageorder = get_order_from_bytes(sizeof(struct ia64_mca_cpu)); \ + ia64_state_log[it].isl_log[IA64_LOG_CURR_INDEX(it)] = \ + (ia64_err_rec_t *)alloc_xenheap_pages(pageorder); \ + ia64_state_log[it].isl_log[IA64_LOG_NEXT_INDEX(it)] = \ + (ia64_err_rec_t *)alloc_xenheap_pages(pageorder); \ + } while(0) +#endif /* XEN */ + #define IA64_LOG_LOCK_INIT(it) spin_lock_init(ia64_state_log[it].isl_lock) #define IA64_LOG_LOCK(it) spin_lock_irqsave(ia64_state_log[it].isl_lock, s) #define IA64_LOG_UNLOCK(it) spin_unlock_irqrestore(ia64_state_log[it].isl_lock,s) @@ -175,6 +197,11 @@ static ia64_state_log_t ia64_state_log[I #define IA64_LOG_CURR_BUFFER(it) (void *)((ia64_state_log[it].isl_log[IA64_LOG_CURR_INDEX(it)])) #define IA64_LOG_COUNT(it) ia64_state_log[it].isl_count +#ifdef XEN +struct list_head sal_queue[IA64_MAX_LOG_TYPES]; +DEFINE_SPINLOCK(sal_queue_lock); +#endif /* XEN */ + /* * ia64_log_init * Reset the OS ia64 log buffer @@ -201,6 +228,7 @@ ia64_log_init(int sal_info_type) memset(IA64_LOG_NEXT_BUFFER(sal_info_type), 0, max_size); } +#ifndef XEN /* * ia64_log_get * @@ -276,15 +304,158 @@ ia64_mca_log_sal_error_record(int sal_in if (rh-severity == sal_log_severity_corrected) ia64_sal_clear_state_info(sal_info_type); } +#else /* XEN */ +/* + * ia64_log_queue + * + * Get the current MCA log from SAL and copy it into the OS log buffer. + * + * Inputs : info_type (SAL_INFO_TYPE_{MCA,INIT,CMC,CPE}) + * Outputs : size(total record length) + *
Re: [Xen-ia64-devel] [PATCH 2/12]MCA handler support for Xen/ia64 TAKE 2
Hi Alex, On Fri, 2006-09-22 at 19:32 +0900, SUZUKI Kazuhiro wrote: [2/12] Add percpu data physical addr mca_asm.S [mca-mca_asm.patch] @@ -221,6 +275,17 @@ 4: ;; srlz.i ;; + // 5. VHPT +#if VHPT_ENABLED + mov r24=VHPT_SIZE_LOG22 + movl r22=VHPT_ADDR + mov r21=IA64_TR_VHPT + ;; + ptr.d r22,r24 + ;; + srlz.d + ;; +#endif Does this need to be inside a #ifdef XEN? @@ -342,6 +422,26 @@ ia64_reload_tr: ;; srlz.d ;; + // 5. VHPT +#if VHPT_ENABLED + mov r24=VHPT_SIZE_LOG22 + movl r22=VHPT_ADDR + mov r21=IA64_TR_VHPT + movl r26=PAGE_KERNEL + ;; + GET_THIS_PADDR(r2, vhpt_paddr) + ;; + ld8 r18=[r2] + ;; + or r23=r18,r26 // construct PA | page properties + mov cr.itir=r24 + mov cr.ifa=r22 + ;; + itr.d dtr[r21]=r23 // wire in new mapping... + ;; + srlz.d + ;; +#endif Same question on this one. Thanks, Yes. I forgot to enclose them with #ifdef XEN. Thanks, KAZ Signed-off-by: Yutaka Ezaki [EMAIL PROTECTED] Signed-off-by: Masaki Kanno [EMAIL PROTECTED] Signed-off-by: Kazuhiro Suzuki [EMAIL PROTECTED] diff -r 3e4fa8b5b245 xen/arch/ia64/linux-xen/mca_asm.S --- a/xen/arch/ia64/linux-xen/mca_asm.S Tue Sep 12 11:43:22 2006 -0600 +++ b/xen/arch/ia64/linux-xen/mca_asm.S Tue Sep 26 10:11:07 2006 +0900 @@ -24,6 +24,9 @@ #include asm/processor.h #include asm/mca_asm.h #include asm/mca.h +#ifdef XEN +#include asm/vhpt.h +#endif /* XEN */ /* * When we get a machine check, the kernel stack pointer is no longer @@ -50,8 +53,7 @@ */ #ifdef XEN #define SAL_TO_OS_MCA_HANDOFF_STATE_SAVE(_tmp) \ - movl_tmp=THIS_CPU(ia64_sal_to_os_handoff_state_addr);; \ - tpa _tmp=_tmp;; \ + GET_THIS_PADDR(_tmp, ia64_sal_to_os_handoff_state_addr);; \ ld8 _tmp=[_tmp];; \ st8 [_tmp]=r1,0x08;;\ st8 [_tmp]=r8,0x08;;\ @@ -72,6 +74,7 @@ st8 [_tmp]=r12,0x08;; \ st8 [_tmp]=r17,0x08;; \ st8 [_tmp]=r18,0x08 +#endif /* XEN */ /* * OS_MCA_TO_SAL_HANDOFF_STATE (SAL 3.0 spec) @@ -101,6 +104,24 @@ * imots_sal_check_ra=Return address to location within SAL_CHECK * */ +#ifdef XEN +#define COLD_BOOT_HANDOFF_STATE(sal_to_os_handoff,os_to_sal_handoff,tmp)\ + movltmp=IA64_MCA_COLD_BOOT; \ + GET_THIS_PADDR(r2,ia64_sal_to_os_handoff_state_addr);; \ + ld8 sal_to_os_handoff=[sal_to_os_handoff];; \ + movlos_to_sal_handoff=ia64_os_to_sal_handoff_state;;\ + dep os_to_sal_handoff = 0, os_to_sal_handoff, 60, 4;; \ + /*DATA_VA_TO_PA(os_to_sal_handoff);;*/ \ + st8 [os_to_sal_handoff]=tmp,8;; \ + ld8 tmp=[sal_to_os_handoff],48;;\ + st8 [os_to_sal_handoff]=tmp,8;; \ + movltmp=IA64_MCA_SAME_CONTEXT;; \ + st8 [os_to_sal_handoff]=tmp,8;; \ + ld8 tmp=[sal_to_os_handoff],-8;;\ + st8 [os_to_sal_handoff]=tmp,8;; \ + ld8 tmp=[sal_to_os_handoff];; \ + st8 [os_to_sal_handoff]=tmp;; +#else /* XEN */ #define COLD_BOOT_HANDOFF_STATE(sal_to_os_handoff,os_to_sal_handoff,tmp)\ movltmp=IA64_MCA_COLD_BOOT; \ movlsal_to_os_handoff=__pa(ia64_sal_to_os_handoff_state); \ @@ -114,13 +135,13 @@ st8 [os_to_sal_handoff]=tmp,8;; \ ld8 tmp=[sal_to_os_handoff];; \ st8 [os_to_sal_handoff]=tmp;; +#endif /* XEN */ #define GET_IA64_MCA_DATA(reg) \ GET_THIS_PADDR(reg, ia64_mca_data) \ ;; \ ld8 reg=[reg] -#endif /* XEN */ .global ia64_os_mca_dispatch .global ia64_os_mca_dispatch_end #ifndef XEN @@ -132,7 +153,40 @@ .text .align 16 -#ifndef XEN +#ifdef XEN +/* + * void set_per_cpu_data(void) + * { + * int i; + * for (i = 0; i 64; i++) { + * if (ia64_mca_tlb_list[i].cr_lid == ia64_getreg(_IA64_REG_CR_LID)) { + * ia64_set_kr(IA64_KR_PER_CPU_DATA, ia64_mca_tlb_list[i].percpu_paddr); + * return; + * } + * } + * while(1); // Endless loop on error + * } + */ +#defineSET_PER_CPU_DATA() \ + LOAD_PHYSICAL(p0,r2,ia64_mca_tlb_list);;\ +
Re: [Xen-ia64-devel] [PATCH 6/12]MCA handler support for Xen/ia64 TAKE 2
Hi Alex, On Fri, 2006-09-22 at 19:33 +0900, SUZUKI Kazuhiro wrote: [6/12] Add sal emulation.[mca-fw_emul.patch] case SAL_GET_STATE_INFO: ... + + spin_lock_irqsave(sal_queue_lock, flags); + ... + ret = smp_call_function_single(e-cpuid, get_state_info_on, arg, 0, 1); Both SAL_GET_STATE_INFO and SAL_CLEAR_STATE_INFO use smp_call_function_single() with interrupts disabled. This is dangerous and can lead to deadlocks. Imagine if two processors called smp_call_function_single() with interrupts disabled, possibly from unrelated code paths. One processor would get the call_lock and send an IPI to the second processor. However, the second processor is spinning trying to get the call_lock with interrupts disabled hang. Thanks, Interrupts disabled state was made only the operation of sal_queue[] like list_entry() and list_del(). Thanks, KAZ Signed-off-by: Yutaka Ezaki [EMAIL PROTECTED] Signed-off-by: Masaki Kanno [EMAIL PROTECTED] Signed-off-by: Kazuhiro Suzuki [EMAIL PROTECTED] diff -r 3e4fa8b5b245 xen/arch/ia64/xen/fw_emul.c --- a/xen/arch/ia64/xen/fw_emul.c Tue Sep 12 11:43:22 2006 -0600 +++ b/xen/arch/ia64/xen/fw_emul.c Thu Sep 28 12:30:34 2006 +0900 @@ -23,6 +23,7 @@ #include linux/efi.h #include asm/pal.h #include asm/sal.h +#include asm/xenmca.h #include public/sched.h #include hpsim_ssc.h @@ -32,6 +33,54 @@ extern unsigned long running_on_sim; +struct sal_mc_params { + u64 param_type; + u64 i_or_m; + u64 i_or_m_val; + u64 timeout; + u64 rz_always; +} sal_mc_params[SAL_MC_PARAM_CPE_INT+1]; + +struct sal_vectors { + u64 vector_type; + u64 handler_addr1; + u64 gp1; + u64 handler_len1; + u64 handler_addr2; + u64 gp2; + u64 handler_len2; +} sal_vectors[SAL_VECTOR_OS_BOOT_RENDEZ+1]; + +struct smp_call_args_t { + u64 type; + u64 ret; + void *data; +}; + +extern spinlock_t sal_queue_lock; + +#defineIA64_SAL_NO_INFORMATION_AVAILABLE -5 + +#if defined(IA64_SAL_DEBUG_INFO) +static const char * const rec_name[] = { MCA, INIT, CMC, CPE }; + +# define IA64_SAL_DEBUG(fmt...)printk(sal_emulator: fmt) +#else +# define IA64_SAL_DEBUG(fmt...) +#endif + +void get_state_info_on(void *data) { + struct smp_call_args_t *arg = data; + + arg-ret = ia64_sal_get_state_info(arg-type, (u64 *)arg-data); +} + +void clear_state_info_on(void *data) { + struct smp_call_args_t *arg = data; + + arg-ret = ia64_sal_clear_state_info(arg-type); +} + struct sal_ret_values sal_emulator (long index, unsigned long in1, unsigned long in2, unsigned long in3, unsigned long in4, unsigned long in5, @@ -102,27 +151,206 @@ sal_emulator (long index, unsigned long } } else - printf(*** CALLED SAL_SET_VECTORS %lu. IGNORED...\n, - in1); + { + if (in1 sizeof(sal_vectors)/sizeof(sal_vectors[0])-1) + BUG(); + sal_vectors[in1].vector_type= in1; + sal_vectors[in1].handler_addr1 = in2; + sal_vectors[in1].gp1= in3; + sal_vectors[in1].handler_len1 = in4; + sal_vectors[in1].handler_addr2 = in5; + sal_vectors[in1].gp2= in6; + sal_vectors[in1].handler_len2 = in7; + } break; case SAL_GET_STATE_INFO: - /* No more info. */ - status = -5; - r9 = 0; + { + sal_queue_entry_t *e; + unsigned long flags; + int size = ia64_sal_get_state_info_size(in1); + static sal_log_record_header_t *record = NULL; + + if (record == NULL) { + unsigned int pageorder; + + pageorder = get_order_from_bytes(size); + record = (sal_log_record_header_t *)alloc_xenheap_pages(pageorder); + } + memset(record, 0, size); + + spin_lock_irqsave(sal_queue_lock, flags); + if (list_empty(sal_queue[in1])) { + sal_log_record_header_t header; + + IA64_SAL_DEBUG(SAL_GET_STATE_INFO(%s) + no sal_queue entry found.\n, rec_name[in1]); + memset(header, 0, sizeof(header)); + if (copy_to_user((void __user *)in3, header, sizeof(header))) { + printk(sal_emulator: SAL_GET_STATE_INFO +
Re: [Xen-ia64-devel] [PATCH 12/12]MCA handler support for Xen/ia64 TAKE 2
Hi Alex, On Fri, 2006-09-22 at 19:33 +0900, SUZUKI Kazuhiro wrote: [12/12] Fix conflicts of typedef of UINT64 and BOOLEAN.[mca-typedef.patch] This is kind of ugly. Can we converge on a single definition for these or at least confine the fix to the ia64 specific code? Thanks, I modified to remove typedef of UINT64/BOOLEAN, and to include linux/acpi.h from vcpu.h instead of using `__TYPEDEF_UINT64__/_BOOLEAN__' macros. Thanks, KAZ Signed-off-by: Yutaka Ezaki [EMAIL PROTECTED] Signed-off-by: Masaki Kanno [EMAIL PROTECTED] Signed-off-by: Kazuhiro Suzuki [EMAIL PROTECTED] diff -r 3e4fa8b5b245 xen/include/asm-ia64/vcpu.h --- a/xen/include/asm-ia64/vcpu.h Tue Sep 12 11:43:22 2006 -0600 +++ b/xen/include/asm-ia64/vcpu.h Tue Sep 26 10:15:40 2006 +0900 @@ -10,9 +10,8 @@ #include asm/ia64_int.h #include xen/types.h #include public/xen.h -typedefunsigned long UINT64; typedefunsigned int UINT; -typedefint BOOLEAN; +#include linux/acpi.h struct vcpu; typedefstruct vcpu VCPU; typedef cpu_user_regs_t REGS; ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH 12/12]MCA handler support for Xen/ia64 TAKE 2
Le Jeudi 28 Septembre 2006 09:11, SUZUKI Kazuhiro a écrit : Hi Alex, On Fri, 2006-09-22 at 19:33 +0900, SUZUKI Kazuhiro wrote: [12/12] Fix conflicts of typedef of UINT64 and BOOLEAN.[mca-typedef.patch] This is kind of ugly. Can we converge on a single definition for these or at least confine the fix to the ia64 specific code? Thanks, I modified to remove typedef of UINT64/BOOLEAN, and to include linux/acpi.h from vcpu.h instead of using `__TYPEDEF_UINT64__/_BOOLEAN__' macros. I think UINT64 and BOOLEAN should be reserved for ACPI. We'd better to remove all UNIT64 and BOOLEAN in xen ia64 code. That's just my opinion but we can do this cleanup later. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11635
Le Jeudi 28 Septembre 2006 08:43, You, Yongkang a écrit : Hi Tristan, We found it for a long time. This issue also can be easily reproduced in latest changeset 11635. I guess it might be the common issue with IA32 side. But IA32 xenU didn't meet the problem with the same operations. Sorry for the wrong subject 11609, I send out yesterday. It should be 11635. Ok, thank you for this info. This bugs look very strange... Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Re: xencomm porting and inline handles
Le Mercredi 27 Septembre 2006 17:10, Hollis Blanchard a écrit : On Wed, 2006-09-27 at 08:19 +0200, Tristan Gingold wrote: Le Mardi 26 Septembre 2006 20:23, Hollis Blanchard a écrit : On Tue, 2006-09-26 at 10:04 +0200, Tristan Gingold wrote: After more work, inline xencomm is not that magic: it doesn't work for modules which are loaded in virtual memory. So I have to use mini xencomm at least for modules. What's the problem with modules? Their text/data isn't physically contiguous, but where exactly is the problem? Inline xencomm only works for physically contiguous area because only the base address is passed. Therefore it doesn't work for modules. I understand that; please explain exactly what about the modules isn't working. For example, the stack used in kernel modules is still physically contiguous, so using stack-allocated data structures should work fine. However, making hypercalls directly using global data structures wouldn't work. However, the inline code is only being used for the hypercalls that could be made early. Is that the problem? Please identify the specific issue(s). Yes, some hypercalls data are global data. Sorry, I was not specific enough! Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel][PATCH][RFC] Task: support huge page RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11460
Hi Anthony. On Wed, Sep 27, 2006 at 09:56:11PM +0800, Xu, Anthony wrote: Currently, memory allocated for domU and VTI-domain is 16K contiguous. That means all huge page TLB entries must be broken into 16K TLB entries. This definitely impact overall performance, for instance, in linux, region 7 is using 16M page size. IA64 is supposed to be used at high end server, many services running on IA64 are using huge page, like Oracle is using 256M page in region 4, if XEN/IA64 still use 16K contiguous physical page, we can image, this can impact performance dramatically. So domU, VTI-domain and dom0 need to support huge page. Attached patch is an experiment to use 16M page on VTI-domain. A very tricky way is used to allocate 16M contiguous memory for VTI-domain, so it's just for reference. Applying this patch, I can see 2%~3% performance gains when running KB on VTI-domain(UP), you may know performance of KB on VTI-domain is not bad:-), that means the improvement is somewhat big. As we know, KB doesn't use 256M, the performance gain is coming from 16M page in region 7, if we run some applications, which use 256M huge page, and then we may get more improvement. I agree with you that supporting tlb insert with large page size and hugetlbfs would be a big gain. In my mind, we need do below things (there may be more) if we want to support huge page. 1. Add an option order in configure file vtiexample.vti. if order=0, XEN/IA64 allocate 16K contiguous memory for domain, if order=1, allocate 32K, and so on. Thus user can chose page size for domain. A fall back path should be implemented in case that large page allocation fails. Or do you propose introducing new page allocator with very large chunk? With order option, page fragmentation should be taken care of. 2. This order option will be past to increase_reservation() function as extent_order argument, increase_reservation() will allocate contiguous memory for domain. 3. There may be some memory blocks, which we also want increase_reservation to allocate for us, such as shared page, or firmware memory for VTI domain etc. So we may need to call increase_reservation() several times to allocate memories with different page size. 4. Per_LP_VHPT may need to be modified to support huge page. Do you mean hash collision? 5. VBD/VNIF may need to be modified to use copy mechanism instead of flipping page. 6. Ballon driver may need to be modified to increase or decrease domain memory by page size not 16K. Magnus, would you like to take this task? Comments are always welcome. Those are my some random thoughts. * Presumably there are two goals - Support one large page size(e.g. 16MB) to map kernel. - Support hugetlbfs whose page size might be different from 16MB. I.e. support three page sizes, normal page size 16KB, kernel mapping page size 16MB and hugetlbfs page size 256MB. I think hugetlbfs support can be addressed specialized way. hugetlbfs * Some specialized path can be implemented to support hugetlbfs. - For domU paravirtualize hugetlbfs for domU. Hook to alloc_fresh_huge_page() in Linux. Then xen/ia64 is aware of large pages. Probably a new flag of the p2m entry, or other data structure might be introduced. For xenLinux, the region number, RGN_HPAGE can be used to check before entering hugetlbfs specialized path. - For domVTI Can the use of hugetlbfs be detected somehow? Probably some Linux-specific heuristic can be used. e.g. check the region, RGN_HPAGE. kernel mapping with large page size. * page fragmentation should be addressed. Both 16KB and 16MB page should be able to co-exist in a same domain. - Allocating large contiguous region might fail. So fall back path should be implemented. - domain should be able to have pages with both page size (16KB and 16MB) for smooth code merge. probably a new bit of the p2m entry, something like _PAGE_HPAGE, would be introduce to distinguish large page from normal page. * paravirtualized driver(VBD/VNIF) This is a really issue. For first prototype it is reasonable to not support page flipping resorting grant table memory copy. There are two kinds of page flipping, page mapping and page transfer. I guess page mapping should be supported somehow assuming only dom0 (or driver domain) maps. We should measure page flipping and memory copy before giving it a try. I have no figures about it. I'm not sure which has better-performance. (I'm biased. I know that vnif analysis on xen/x86. It said memory copy was cheaper on x86 than page flipping...) If dom0 does only DMA, I/O request can be completed without copy and tlb flush for VBD with tlb tracking patch. Page transfer is difficult. I'm not sure that it's worth while to support page transfer because I'm suffering in optimize it. Another approach is * increase xen page size. Probably simply increasing page size
Re: [Xen-ia64-devel] [PATCH 1/12]MCA handler support for Xen/ia64 TAKE 2
Kazuhiro == SUZUKI Kazuhiro [EMAIL PROTECTED] writes: +#ifndef XEN static struct timer_list cpe_poll_timer; static struct timer_list cmc_poll_timer; +#else /* XEN */ +static struct timer cpe_poll_timer; +static struct timer cmc_poll_timer; +#endif Hi Kazuhiro, Any chance we could start using sane names in Xen that match the Linux names so we don't have to end up with a ton of #ifdef XEN changes like that? It makes it really difficult to read and maintain the code long term. Cheers, Jes ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH 12/12]MCA handler support for Xen/ia64 TAKE 2
Tristan == Tristan Gingold [EMAIL PROTECTED] writes: Tristan I think UINT64 and BOOLEAN should be reserved for ACPI. We'd Tristan better to remove all UNIT64 and BOOLEAN in xen ia64 code. Tristan That's just my opinion but we can do this cleanup later. That one gets my vote too :) We should really try and stick to consistent types and match them with Linux. The ACPI ones solely exist so the ACPI code can be shared with that other operating system that nobody likes to talk about. Cheers, Jes ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609
HI Yoshinori, Thanks for reproduce. But if want to create xenU successfully again, I have to reboot the machine. :) Best Regards, Yongkang (Kangkang) 永康 -Original Message- From: Yoshinori Katase [mailto:[EMAIL PROTECTED] Sent: 2006年9月28日 16:47 To: You, Yongkang; xen-ia64-devel@lists.xensource.com Subject: Re: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609 Hi Yongkang, I confirmed the phenomenon you reported.(11635:f34e37d0742d) It is very easy to reproduce. But not have to reboot machine to recover. [EMAIL PROTECTED] ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 988 3 r-45.3 [EMAIL PROTECTED] xen]# (xm create rhel4.cs11609 );xm create rhel4.cs11609-2 [1] 6045 [EMAIL PROTECTED] xen]# Using config file rhel4.cs11609-2. Using config file rhel4.cs11609. Started domain rhel4.cs11609 Started domain rhel4.cs11609-2 [1]+ Donexm create rhel4.cs11609-2 [EMAIL PROTECTED] xen]# xm list Error: Device 0 not connected [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 988 3 r-69.5 Zombie-rhel4.cs11609 2 1024 1 cd 6.1 Zombie-rhel4.cs11609 4 1024 1 cd 0.0 Zombie-rhel4.cs11609-2 1 1024 1 cd 6.1 Zombie-rhel4.cs11609-2 3 1024 1 cd 0.1 [EMAIL PROTECTED] xen]# xend stop [EMAIL PROTECTED] xen]# xend start [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 988 3 r-80.0 [EMAIL PROTECTED] xen]# (xm create rhel4.cs11609 );xm create rhel4.cs11609-2 [1] 7978 [EMAIL PROTECTED] xen]# Using config file rhel4.cs11609. Using config file rhel4.cs11609-2. Started domain rhel4.cs11609 Started domain rhel4.cs11609-2 [1]+ Donexm create rhel4.cs11609-2 [EMAIL PROTECTED] xen]# xm list Error: Device 0 not connected [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 988 3 r-98.9 Zombie-rhel4.cs11609 5 1024 1 cd 0.1 Zombie-rhel4.cs11609 7 1024 1 cd 0.1 Zombie-rhel4.cs11609-2 8 1024 1 cd 0.1 Zombie-rhel4.cs11609-2 6 1024 1 cd 0.4 Best Regards, Katase - Original Message - From: You, Yongkang [EMAIL PROTECTED] To: You, Yongkang [EMAIL PROTECTED]; xen-ia64-devel@lists.xensource.com Sent: Wednesday, September 27, 2006 7:05 PM Subject: RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609 The 2 XenU coexisting testing is easy to fail. I try 3 times to create 2 XenU domains at the same time (xm create config1 ) ; xm create config2 . The first 2 times are pass. But the 3rd time. XenU creating failed. After that, xm list can not list domains and will report Error: Device 768 not connected Use xend stop and xend start could recover xm list. But if try to create 2 XenU domains again, there will be 4 Zombie-Domains in xm list. It is same with the bug http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=747 Best Regards, Yongkang (Kangkang) 永康 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of You, Yongkang Sent: 2006年9月27日 15:05 To: xen-ia64-devel@lists.xensource.com Subject: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609 Xen/IA64 Healthiness Report 2 XenU coexisting case failed. If try to create 2 XenU at the same time, Xend will be broken, e.g. xm list just show xm help information. Have to reboot machine to recover. We will report a new bug for this. Testing Environment: Platform: Tiger4 Processor: Itanium 2 Processor Logic Processors number: 8 (2 processors with Due Core) PAL version: 8.15 Service OS: RHEL4u3 IA64 SMP with 2 VCPUs VTI Guest OS: RHEL4u2 RHEL4u3 XenU Guest OS: RHEL4u2 Xen IA64 Unstable tree: 11635:f34e37d0742d Xen Schedule: credit VTI Guest Firmware MD5: ea1815932b91e5a6481bb64468e50254 Summary Test Report: - Total cases: 16 Passed:15 Failed: 1 Detailed test cases info: - 1. One_VTI_without_vifpass 2. One_VTI pass 3. One_XenU pass 4. Two_VTI_Coexist pass 5. One_VTI_and_One_XenU pass 6. Two_XenU_Coexist __FAIL__ 7. One_VTI_4096M pass 8. VTI_Network pass 9. XenU_Network pass 10. One_SMP_XenU pass 11. One_SMP_VTIpass 12. UP_VTI_LTP
Re: [Xen-ia64-devel] sparsemem/discontig?
Tristan Gingold wrote: Le Mardi 26 Septembre 2006 10:24, Jes Sorensen a écrit : Hmmm, so why did it make no difference whether I changed those addresses? This is not an early crash. Xen boots rather far before crashing. Any idea if there's some hard coded places that I need to track down? I don't know the altix at all. It seems the memory is very sparse. Have you post the boot log ? Salut Tristan, Looks like someone got the physical address wrong when he changed the define (also known as error code 40, located 40cm from the screen). After I set it to the correct address it gets much further, however this underlines the issue of having to fix relocation of the physical addresses in TLB handlers etc. Guess I know what I'll be doing for the next couple of days :) Cheers, Jes ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609
Hi, Yongkang, I came across the same message, too. The changeset were 11609 and 11456. The number of vcpus in a guest seems to have some relevance since I see this bug almost always with vcpus=4 in XenU configuration, while I have no problem with vcpus=1, 2 and 3. In addition, I could biring up XenU with vcpus=3 just after the failure occurred with vcpus=4. The command line and the message: # xm create /etc/xen/rhel4PAR-xx.xx.xx.xx-vcpu4 -c Using config file /etc/xen/rhel4PAR-xx.xx.xx.xx-vcpu4. Error: Device 768 (vbd) could not be connected. Backend device not found. A detailed configurations: [XenU config] # grep -v '^#' /etc/xen/rhel4PAR-xx.xx.xx.xx-vcpu4 |grep -v '^$' kernel = /boot/efi/efi/xen/vmlinuz-2.6.16.29-xen-cs11609-inakoshi ramdisk = /boot/efi/efi/xen/initrd-2.6.16.29-xen.img-cs11609-inakoshi memory = 512 name = rhel4PAR-179.69 vcpus = 4 vif = [ '' ] disk = [ 'file:/xen/image/rhel4u2-xx.xx.xx.xx-5GB-inakoshi.img,hda,w' ] root = /dev/hda2 ro extra = nomca console=tty0 root=/dev/hda2 3 [elilo.conf] prompt timeout=20 default=xen-cs11609-inakoshi relocatable image=vmlinuz-2.6.9-42.EL label=linux initrd=initrd-2.6.9-42.EL.img read-only append=rhgb quiet root=LABEL=/ image=vmlinuz-2.6.16.29-xen-cs11609-inakoshi label=xen-cs11609-inakoshi vmm=xen.gz-cs11609-inakoshi initrd=initrd-2.6.16.29-xen.img-cs11609-inakoshi read-only append=com2=115200,8n1 console=com2 sched=credit tbuf_size=128 maxcpus=4 dom0_max_vcpus=4 -- nomca dom0_vcpus=4 console=tty0 console=ttyS1,115200,8n1 rhgb root=LABEL=/ ## Note: The append line below also caused the bug. #append=com2=115200,8n1 console=com2 sched=credit tbuf_size=128 dom0_vcpus_pin -- nomca console=tty0 console=ttyS1,115200,8n1 rhgb root=LABEL=/ [xm info] host : xx.xx.xx.xx release: 2.6.16.29-xen version: #1 SMP Thu Sep 28 14:26:53 JST 2006 machine: ia64 nr_cpus: 4 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 2 threads_per_core : 1 cpu_mhz: 1396 hw_caps: :::::::: total_memory : 4070 free_memory: 3503 xen_major : 3 xen_minor : 0 xen_extra : -unstable xen_caps : xen-3.0-ia64 hvm-3.0-ia64 xen_pagesize : 16384 platform_params: virt_start=0xe800 xen_changeset : Sun Sep 24 14:55:57 2006 -0600 11609:7b250cf49e50 cc_compiler: gcc version 3.4.5 cc_compile_by : inakoshi cc_compile_domain : xx.xx.xx.xx cc_compile_date: Thu Sep 28 15:13:08 JST 2006 xend_config_format : 2 Regards, Hiroya You, Yongkang wrote: The 2 XenU coexisting testing is easy to fail. I try 3 times to create 2 XenU domains at the same time (xm create config1 ) ; xm create config2 . The first 2 times are pass. But the 3rd time. XenU creating failed. After that, xm list can not list domains and will report Error: Device 768 not connected Use xend stop and xend start could recover xm list. But if try to create 2 XenU domains again, there will be 4 Zombie-Domains in xm list. It is same with the bug http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=747 Best Regards, Yongkang (Kangkang) 永康 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of You, Yongkang Sent: 2006年9月27日 15:05 To: xen-ia64-devel@lists.xensource.com Subject: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609 Xen/IA64 Healthiness Report 2 XenU coexisting case failed. If try to create 2 XenU at the same time, Xend will be broken, e.g. xm list just show xm help information. Have to reboot machine to recover. We will report a new bug for this. Testing Environment: Platform: Tiger4 Processor: Itanium 2 Processor Logic Processors number: 8 (2 processors with Due Core) PAL version: 8.15 Service OS: RHEL4u3 IA64 SMP with 2 VCPUs VTI Guest OS: RHEL4u2 RHEL4u3 XenU Guest OS: RHEL4u2 Xen IA64 Unstable tree: 11635:f34e37d0742d Xen Schedule: credit VTI Guest Firmware MD5: ea1815932b91e5a6481bb64468e50254 Summary Test Report: - Total cases: 16 Passed:15 Failed: 1 Detailed test cases info: - 1. One_VTI_without_vifpass 2. One_VTI pass 3. One_XenU pass 4. Two_VTI_Coexist pass 5. One_VTI_and_One_XenU pass 6. Two_XenU_Coexist __FAIL__ 7. One_VTI_4096M pass 8. VTI_Network pass 9. XenU_Network pass 10. One_SMP_XenU pass 11. One_SMP_VTIpass 12. UP_VTI_LTP pass
Re: [Xen-ia64-devel] sparsemem/discontig?
Le Jeudi 28 Septembre 2006 11:50, Jes Sorensen a écrit : Tristan Gingold wrote: Le Mardi 26 Septembre 2006 10:24, Jes Sorensen a écrit : Hmmm, so why did it make no difference whether I changed those addresses? This is not an early crash. Xen boots rather far before crashing. Any idea if there's some hard coded places that I need to track down? I don't know the altix at all. It seems the memory is very sparse. Have you post the boot log ? Salut Tristan, Looks like someone got the physical address wrong when he changed the define (also known as error code 40, located 40cm from the screen). After I set it to the correct address it gets much further, however this underlines the issue of having to fix relocation of the physical addresses in TLB handlers etc. Guess I know what I'll be doing for the next couple of days :) Changing the physical loading address shouldn't that hard. I'd suggest you *not* working on relocation yet. According to page_alloc.c, the allocator needs a bitmap for memory. This bitmap is not sparse. I suppose this is a major issue for Altix to be fixed. Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel][PATCH][RFC] Task: support huge page RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11460
Le Jeudi 28 Septembre 2006 10:07, Isaku Yamahata a écrit : Hi Anthony. On Wed, Sep 27, 2006 at 09:56:11PM +0800, Xu, Anthony wrote: Currently, memory allocated for domU and VTI-domain is 16K contiguous. That means all huge page TLB entries must be broken into 16K TLB entries. This definitely impact overall performance, for instance, in linux, region 7 is using 16M page size. IA64 is supposed to be used at high end server, many services running on IA64 are using huge page, like Oracle is using 256M page in region 4, if XEN/IA64 still use 16K contiguous physical page, we can image, this can impact performance dramatically. So domU, VTI-domain and dom0 need to support huge page. Attached patch is an experiment to use 16M page on VTI-domain. A very tricky way is used to allocate 16M contiguous memory for VTI-domain, so it's just for reference. Applying this patch, I can see 2%~3% performance gains when running KB on VTI-domain(UP), you may know performance of KB on VTI-domain is not bad:-), that means the improvement is somewhat big. As we know, KB doesn't use 256M, the performance gain is coming from 16M page in region 7, if we run some applications, which use 256M huge page, and then we may get more improvement. I agree with you that supporting tlb insert with large page size and hugetlbfs would be a big gain. In my mind, we need do below things (there may be more) if we want to support huge page. 1. Add an option order in configure file vtiexample.vti. if order=0, XEN/IA64 allocate 16K contiguous memory for domain, if order=1, allocate 32K, and so on. Thus user can chose page size for domain. A fall back path should be implemented in case that large page allocation fails. Or do you propose introducing new page allocator with very large chunk? With order option, page fragmentation should be taken care of. 2. This order option will be past to increase_reservation() function as extent_order argument, increase_reservation() will allocate contiguous memory for domain. 3. There may be some memory blocks, which we also want increase_reservation to allocate for us, such as shared page, or firmware memory for VTI domain etc. So we may need to call increase_reservation() several times to allocate memories with different page size. 4. Per_LP_VHPT may need to be modified to support huge page. Do you mean hash collision? 5. VBD/VNIF may need to be modified to use copy mechanism instead of flipping page. 6. Ballon driver may need to be modified to increase or decrease domain memory by page size not 16K. Magnus, would you like to take this task? Comments are always welcome. Those are my some random thoughts. * Presumably there are two goals - Support one large page size(e.g. 16MB) to map kernel. - Support hugetlbfs whose page size might be different from 16MB. I.e. support three page sizes, normal page size 16KB, kernel mapping page size 16MB and hugetlbfs page size 256MB. I think hugetlbfs support can be addressed specialized way. hugetlbfs * Some specialized path can be implemented to support hugetlbfs. - For domU paravirtualize hugetlbfs for domU. Hook to alloc_fresh_huge_page() in Linux. Then xen/ia64 is aware of large pages. Probably a new flag of the p2m entry, or other data structure might be introduced. For xenLinux, the region number, RGN_HPAGE can be used to check before entering hugetlbfs specialized path. - For domVTI Can the use of hugetlbfs be detected somehow? Probably some Linux-specific heuristic can be used. e.g. check the region, RGN_HPAGE. kernel mapping with large page size. * page fragmentation should be addressed. Both 16KB and 16MB page should be able to co-exist in a same domain. - Allocating large contiguous region might fail. So fall back path should be implemented. - domain should be able to have pages with both page size (16KB and 16MB) for smooth code merge. probably a new bit of the p2m entry, something like _PAGE_HPAGE, would be introduce to distinguish large page from normal page. * paravirtualized driver(VBD/VNIF) This is a really issue. For first prototype it is reasonable to not support page flipping resorting grant table memory copy. There are two kinds of page flipping, page mapping and page transfer. I guess page mapping should be supported somehow assuming only dom0 (or driver domain) maps. We should measure page flipping and memory copy before giving it a try. I have no figures about it. I'm not sure which has better-performance. (I'm biased. I know that vnif analysis on xen/x86. It said memory copy was cheaper on x86 than page flipping...) If dom0 does only DMA, I/O request can be completed without copy and tlb flush for VBD with tlb tracking patch. Page transfer is difficult. I'm not sure that it's worth while to support
Re: [Xen-ia64-devel][PATCH][RFC] Task: support huge page RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11460
On Thu, Sep 28, 2006 at 12:59:17PM +0200, Tristan Gingold wrote: If we want to support linux page-size != Xen page-size some similar issues are encountered. I therefore suppose the both features should be worked together. Yes. Some of them impily Linux page-size != Xen page-size. -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] sparsemem/discontig?
Tristan Gingold wrote: Changing the physical loading address shouldn't that hard. I'd suggest you *not* working on relocation yet. Hi Tristan, Changing the address for loading works, except that our machines are not all the same and I'd have to set it individually for each one. I'd rather start looking at the relocation so we get an infrastructure for that sorted. According to page_alloc.c, the allocator needs a bitmap for memory. This bitmap is not sparse. I suppose this is a major issue for Altix to be fixed. Yup, I hit that one too, takes forever to boot because of this :( Cheers, Jes ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] sparsemem/discontig?
Le Jeudi 28 Septembre 2006 13:53, Jes Sorensen a écrit : Tristan Gingold wrote: Changing the physical loading address shouldn't that hard. I'd suggest you *not* working on relocation yet. Hi Tristan, Changing the address for loading works, except that our machines are not all the same and I'd have to set it individually for each one. I'd rather start looking at the relocation so we get an infrastructure for that sorted. Ok, so this is more urgent that I previously thought. I am working on a patch which change Xen mapping making it much more flexible. I will add the relocation feature. According to page_alloc.c, the allocator needs a bitmap for memory. This bitmap is not sparse. I suppose this is a major issue for Altix to be fixed. Yup, I hit that one too, takes forever to boot because of this :( You'd better to work on this issue as SGI is the only one which such features! Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] sparsemem/discontig?
Tristan Gingold wrote: Changing the address for loading works, except that our machines are not all the same and I'd have to set it individually for each one. I'd rather start looking at the relocation so we get an infrastructure for that sorted. Ok, so this is more urgent that I previously thought. Yeah :( Yup, I hit that one too, takes forever to boot because of this :( You'd better to work on this issue as SGI is the only one which such features! I think it's an issue for HP too when the Superdome is configured to use sparse memory, but yeah, I'll try and look into this one ... first I want this thing to boot :) Cheers, Jes ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] Re: ia64 kexec: xen - linux
On 9/27/06, Tristan Gingold [EMAIL PROTECTED] wrote: Le Mercredi 27 Septembre 2006 11:36, Magnus Damm a écrit : On 9/15/06, Horms [EMAIL PROTECTED] wrote: Hi, as some of you may be aware I am working on porting kexec to xen/ia64. I have made a reasoble ammount of progress to this end. I'll try and get a new patch set on xen-devel some time next week. However I have a problem that I need some ideas on how to solve. At the moment when kexecing from xen to linux the boot halts on a call to efi_gettimeofday(), or more specifically efi.get_time. I'm assuming that this is more or less the first efi runtime call that is made, and that it is halting because of a discrepancy in the virtual mapping set up by efi.set_virtual_address_map(). The problem as I see it is that linux uses a page_offset that covers the most significant 3 bits, wherase xen uses the first 4. The unfortunate thing is that efi.set_virtual_address_map() can only be called once, and I don't think its possible to change the mappings at all once its been called. One idea that I had was to make sure that the efi calls are always made in real mode, and never call efi.set_virtual_address_map() at all - efi calls have to be made using virtual addressing after efi.set_virtual_address_map() is called. But can this work? Another idea from my colleague Magnus was to map the efi runtime calls into some area of memory that is agreed upon by both Linux and Xen (and any other kexec-able OS/hypervisor). This seems to be tedious at best. To clarify this a bit, my plan was to extend the bootloader to provide some kind resident efi filter code. This code should act as a filter for all efi run time calls including the dreaded now use-once set_virtual_address_map() function. The most important task for this code would be to support an unlimited number of set_virtual_address_map() calls. I suspect this can be done by always calling the efi functions from real mode. This technique does however require switching back and forth to real mode, and I'm not sure how well that will work out with NMI:s and data that crosses page boundaries. A first step would probably be to try to convert Linux into calling efi runtime functions from real mode. If that works well then the code can be broken out, made resident and placed into the bootloader. Linux and xen call efi in real mode if set_virtual_address_map fails. You may add an option in both xen and linux to force calling efi in real mode. This should be really simple and you will be able to make progress. Oh, is that so? I have not looked at the code yet, only discussed this issue with Horms so far. But that's good news. Thanks for the suggestion! / magnus ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] RFC: nested Xen
Hi, This is a very old patch I have updated and I'd like to submit soon. Basically it makes VMM memory space much more flexible. With this patch, you can compile a Xen with CONFIG_XOX defined and run this Xen within VTi! I now have to update this patch to add relocation feature (required at least for Altix). Tristan. diff -r 869cc1f44e52 xen/arch/ia64/Makefile --- a/xen/arch/ia64/Makefile Thu Sep 28 12:47:49 2006 +0200 +++ b/xen/arch/ia64/Makefile Thu Sep 28 13:33:05 2006 +0200 @@ -68,7 +68,7 @@ asm-xsi-offsets.s: asm-xsi-offsets.c $(H || ln -sf ../../../arch/x86/hvm/vioapic.c $(BASEDIR)/arch/ia64/vmx/hvm_vioapic.c # I'm sure a Makefile wizard would know a better way to do this -xen.lds.s: xen/xen.lds.S +xen.lds.s: xen/xen.lds.S $(HDRS) $(CC) -E $(CPPFLAGS) -P -DXEN $(AFLAGS) \ -o xen.lds.s xen/xen.lds.S diff -r 869cc1f44e52 xen/arch/ia64/linux-xen/entry.S --- a/xen/arch/ia64/linux-xen/entry.S Thu Sep 28 12:47:49 2006 +0200 +++ b/xen/arch/ia64/linux-xen/entry.S Thu Sep 28 13:33:05 2006 +0200 @@ -199,7 +199,7 @@ GLOBAL_ENTRY(ia64_switch_to) movl r27=THIS_CPU(cpu_kr)+IA64_KR_CURRENT_STACK_OFFSET;; ld8 r27=[r27] adds r21=IA64_TASK_THREAD_KSP_OFFSET,in0 - dep r20=0,in0,60,4 // physical address of next + XEN_VA_TO_PA(r20,in0) // physical address of next #else mov r27=IA64_KR(CURRENT_STACK) adds r21=IA64_TASK_THREAD_KSP_OFFSET,in0 diff -r 869cc1f44e52 xen/arch/ia64/linux-xen/head.S --- a/xen/arch/ia64/linux-xen/head.S Thu Sep 28 12:47:49 2006 +0200 +++ b/xen/arch/ia64/linux-xen/head.S Thu Sep 28 13:33:05 2006 +0200 @@ -344,7 +344,7 @@ 1: // now we are in virtual mode ;; or r18=r17,r18 #ifdef XEN - dep r2=-1,r3,60,4 // IMVA of task + XEN_PA_TO_VA(r2,r3) // IMVA of task #else dep r2=-1,r3,61,3 // IMVA of task #endif @@ -397,7 +397,7 @@ 1: // now we are in virtual mode mov ar.rsc=0x3 // place RSE in eager mode #ifdef XEN -(isBP) dep r28=-1,r28,60,4 // make address virtual +(isBP) XEN_PA_TO_VA(r28,r28) // make addr virtual #else (isBP) dep r28=-1,r28,61,3 // make address virtual #endif diff -r 869cc1f44e52 xen/arch/ia64/linux-xen/pal.S --- a/xen/arch/ia64/linux-xen/pal.S Thu Sep 28 12:47:49 2006 +0200 +++ b/xen/arch/ia64/linux-xen/pal.S Thu Sep 28 13:33:05 2006 +0200 @@ -14,7 +14,9 @@ #include asm/asmmacro.h #include asm/processor.h - +#ifdef XEN +#include asm/xensystem.h +#endif .data pal_entry_point: data8 ia64_pal_default_handler @@ -167,7 +169,7 @@ 1: { ;; mov loc4=ar.rsc // save RSE configuration #ifdef XEN - dep.z loc2=loc2,0,60 // convert pal entry point to physical + XEN_VA_TO_PA(loc2,loc2) // convert pal entry point to physical #else // XEN dep.z loc2=loc2,0,61 // convert pal entry point to physical #endif // XEN @@ -230,7 +232,7 @@ 1: { ;; mov loc4=ar.rsc // save RSE configuration #ifdef XEN - dep.z loc2=loc2,0,60 // convert pal entry point to physical + XEN_VA_TO_PA(loc2,loc2) // convert pal entry point to physical #else // XEN dep.z loc2=loc2,0,61 // convert pal entry point to physical #endif // XEN diff -r 869cc1f44e52 xen/arch/ia64/vmx/vmx_entry.S --- a/xen/arch/ia64/vmx/vmx_entry.S Thu Sep 28 12:47:49 2006 +0200 +++ b/xen/arch/ia64/vmx/vmx_entry.S Thu Sep 28 13:33:05 2006 +0200 @@ -599,10 +599,8 @@ 1: { ;; tpa loc2 = loc2 // get physical address of per cpu date ;; -dep loc3 = 0,in1,60,4 // get physical address of shared_info -dep loc4 = 0,in2,60,4 // get physical address of shared_arch_info -dep loc5 = 0,in3,60,4 // get physical address of guest_vhpt -dep loc6 = 0,in4,60,4 // get physical address of pal code +XEN_VA_TO_PA(loc5,in3) // get physical address of guest_vhpt +XEN_VA_TO_PA(loc6,in4) // get physical address of pal code ;; mov loc7 = psr // save psr ;; diff -r 869cc1f44e52 xen/arch/ia64/vmx/vmx_init.c --- a/xen/arch/ia64/vmx/vmx_init.c Thu Sep 28 12:47:49 2006 +0200 +++ b/xen/arch/ia64/vmx/vmx_init.c Thu Sep 28 13:33:05 2006 +0200 @@ -93,6 +93,13 @@ identify_vmx_feature(void) goto no_vti; } +/* Be sure the hypervisor is protected by VTi VA. */ +if (XEN_VA_SZ != 59) { +printk(VTi disabled due to CONFIG_XEN_VA_SZ != 59 (value is %d)\n, + CONFIG_XEN_VA_SZ); +goto no_vti; +} + /* Does xen has ability to decode itself? */ if (!(vp_env_info VP_OPCODE)) printk(WARNING: no opcode provided from hardware(%lx)!!!\n, vp_env_info); diff -r 869cc1f44e52 xen/arch/ia64/vmx/vmx_ivt.S --- a/xen/arch/ia64/vmx/vmx_ivt.S Thu Sep 28 12:47:49 2006 +0200 +++ b/xen/arch/ia64/vmx/vmx_ivt.S Thu Sep 28 13:33:05 2006 +0200 @@ -300,7 +300,7 @@ vmx_alt_itlb_miss_1: movl r19=(((1 IA64_MAX_PHYS_BITS) - 1) ~0xfff) ;; and r19=r19,r16 // clear ed, reserved bits, and PTE control bits -shr.u r18=r16,55// move address bit 59 to bit 4 +shr.u r18=r16,XEN_VA_QUAD_SZ-4// move address bit nocache to bit 4 ;; and r18=0x10,r18// bit
[Xen-ia64-devel] Re: blkbk/netbk modules fail to load on fedora xen/ia64
Akio Takebe wrote: [Thu Sep 28 2006, 02:44:54AM EDT] Hi, Aron and all netbk/xennet modules fail to load by using xen-ia64-unstable again. This issue is caused by the following patches. http://xenbits.xensource.com/ext/xen-ia64-unstable.hg?cs=c757ebffd500 I guess it's no surprise this would continue to break... :-| We can probably resolve this issue. 1. Tristan's xencomm patches is applied. (I test by using xen-ia64-unstable, but these patches have some issue.) What issues are you seeing? They seem to be working well for me, but I haven't tried on Fedora's kernel yet, which is my next goal. 2. revert the below patch. (I have not test yet.) 3. built in these modules. (I have not test yet.) I think the best way is 1. But Tristan's patches still have some issues. What do you think about this? I agree with you. If possible, let's fix the issues in xencomm and get them added to Fedora. Aron ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] Re: blkbk/netbk modules fail to load on fedora xen/ia64
Aron Griffis wrote: [Thu Sep 28 2006, 10:31:41AM EDT] What issues are you seeing? They seem to be working well for me, but I haven't tried on Fedora's kernel yet, which is my next goal. Nevermind this question... I just caught up on xen-ia64-devel emails, so I hadn't seen your report to Tristan. I see it now, and also Tristan's updated patch, so I'll test that on Fedora. Regards, Aron ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] PATCH: xencomm [2]
Hi, Tristan I tried to test your patches. And I don't meet the network issue. good work!!! I'll try to test my network load test. I had some unaligned access message. For your infomation, I show the log. PING 10.124.36.60 (10.124.36.60) 56(84) bytes of data. kernel unaligned access to 0xe00019247a1e, ip=0xa0010093d681 kernel unaligned access to 0xe00019247a1e, ip=0xa0010093d751 kernel unaligned access to 0xe00019247a1e, ip=0xa0010093d800 kernel unaligned access to 0xe00019247a2e, ip=0xa0010093da10 kernel unaligned access to 0xe00019247a1e, ip=0xa0010093da90 64 bytes from 10.124.36.60: icmp_seq=0 ttl=126 time=42.7 ms 64 bytes from 10.124.36.60: icmp_seq=1 ttl=126 time=2.31 ms 64 bytes from 10.124.36.60: icmp_seq=2 ttl=126 time=2.28 ms 64 bytes from 10.124.36.60: icmp_seq=3 ttl=126 time=2.29 ms 64 bytes from 10.124.36.60: icmp_seq=4 ttl=126 time=2.30 ms 64 bytes from 10.124.36.60: icmp_seq=5 ttl=126 time=2.29 ms kernel unaligned access to 0xe0001918d21e, ip=0xa0010093d681 kernel unaligned access to 0xe0001918d21e, ip=0xa0010093d751 kernel unaligned access to 0xe0001918d21e, ip=0xa0010093d800 kernel unaligned access to 0xe0001918d22e, ip=0xa0010093da10 kernel unaligned access to 0xe0001918ca1e, ip=0xa0010093d681 64 bytes from 10.124.36.60: icmp_seq=6 ttl=126 time=32.6 ms 64 bytes from 10.124.36.60: icmp_seq=7 ttl=126 time=2.24 ms 64 bytes from 10.124.36.60: icmp_seq=8 ttl=126 time=2.38 ms 64 bytes from 10.124.36.60: icmp_seq=9 ttl=126 time=2.26 ms 64 bytes from 10.124.36.60: icmp_seq=10 ttl=126 time=2.25 ms 64 bytes from 10.124.36.60: icmp_seq=11 ttl=126 time=2.24 ms Best Regards, Akio Takebe Hi, thanks to Alex and Akio for more torough testing. I have fixed the xentop issue. I think I have also fixed the network issue: during the tests I never hit it. My tests were booting dom0+2*dom_U (using netbk, blkbk and netloop as modules), and doing ping+wget from domU_1. Tristan. ---text/plain--- ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[Xen-ia64-devel] Re: PATCH: xencomm [2]
On Thu, 2006-09-28 at 13:33 +0200, Tristan Gingold wrote: Hi, thanks to Alex and Akio for more torough testing. I have fixed the xentop issue. I think I have also fixed the network issue: during the tests I never hit it. My tests were booting dom0+2*dom_U (using netbk, blkbk and netloop as modules), and doing ping+wget from domU_1. Hi Tristan, Thanks, looks a lot better. I've completed hundreds of reboot cycles now. The next problem is that domain saving is broken. I get this when trying to save a domain: (XEN) xencomm_paddr_to_maddr: called with bad memory address: 0xe0003a527e20 - iip=a001000690a0 The save file never gets bigger than appended below. I also get a few new warnings when creating VTi domains, but they don't seem to hurt anything: privcmd_hypercall: unknown hcall (34) xencomm_privcmd_memory_op: unknown memory op 6 Thanks, Alex -- Alex Williamson HP Open Source Linux Org. Save file contents: 000: 4c69 6e75 7847 7565 7374 5265 636f 7264 LinuxGuestRecord 010: 02d7 2864 6f6d 6169 6e20 2864 6f6d (domain (dom 020: 6964 2031 2920 2875 7569 6420 3135 3061 id 1) (uuid 150a 030: 6433 6665 2d61 3065 612d 3633 3039 2d38 d3fe-a0ea-6309-8 040: 6162 342d 6466 6536 6362 6133 3165 3832 ab4-dfe6cba31e82 050: 2920 2876 6370 7573 2031 2920 2876 6370 ) (vcpus 1) (vcp 060: 755f 6176 6169 6c20 3129 2028 6370 755f u_avail 1) (cpu_ 070: 7765 6967 6874 2031 2e30 2920 286d 656d weight 1.0) (mem 080: 6f72 7920 3130 3234 2920 2873 6861 646f ory 1024) (shado 090: 775f 6d65 6d6f 7279 2030 2920 286d 6178 w_memory 0) (max 0a0: 6d65 6d20 3130 3234 2920 2866 6561 7475 mem 1024) (featu 0b0: 7265 7320 2920 286e 616d 6520 7361 7267 res ) (name sarg 0c0: 6531 2920 286f 6e5f 706f 7765 726f e1) (on_poweroff 0d0: 2064 6573 7472 6f79 2920 286f 6e5f 7265 destroy) (on_re 0e0: 626f 6f74 2072 6573 7461 7274 2920 286f boot restart) (o 0f0: 6e5f 6372 6173 6820 7265 7374 6172 7429 n_crash restart) 100: 2028 696d 6167 6520 286c 696e 7578 2028 (image (linux ( 110: 6b65 726e 656c 202f 6f70 742f 7865 6e2f kernel /opt/xen/ 120: 766d 6c69 6e75 7829 2028 726f 6f74 2027 vmlinux) (root ' 130: 2f64 6576 2f68 6461 3120 726f 2729 2028 /dev/hda1 ro') ( 140: 6172 6773 2027 636f 6e73 6f6c 653d 7474 args 'console=tt 150: 7931 2027 2929 2920 2864 6576 6963 6520 y1 '))) (device 160: 2876 6966 2028 6261 636b 656e 6420 3029 (vif (backend 0) 170: 2028 7363 7269 7074 2076 6966 2d62 7269 (script vif-bri 180: 6467 6529 2028 6d61 6320 3030 3a31 363a dge) (mac 00:16: 190: 3365 3a31 333a 6638 3a39 3929 2929 2028 3e:13:f8:99))) ( 1a0: 6465 7669 6365 2028 7662 6420 2862 6163 device (vbd (bac 1b0: 6b65 6e64 2030 2920 2864 6576 2068 6461 kend 0) (dev hda 1c0: 313a 6469 736b 2920 2875 6e61 6d65 2066 1:disk) (uname f 1d0: 696c 653a 2f6f 7074 2f78 656e 2f73 6172 ile:/opt/xen/sar 1e0: 6765 3129 2028 6d6f 6465 2077 2929 2920 ge1) (mode w))) 1f0: 2864 6576 6963 6520 2876 6264 2028 6261 (device (vbd (ba 200: 636b 656e 6420 3029 2028 6465 7620 6864 ckend 0) (dev hd 210: 623a 6469 736b 2920 2875 6e61 6d65 2066 b:disk) (uname f 220: 696c 653a 2f6f 7074 2f78 656e 2f73 7065 ile:/opt/xen/spe 230: 7729 2028 6d6f 6465 2077 2929 2920 2873 w) (mode w))) (s 240: 7461 7465 202d 622d 2d2d 2d29 2028 7368 tate -b) (sh 250: 7574 646f 776e 5f72 6561 736f 6e20 706f utdown_reason po 260: 7765 726f 2920 2863 7075 5f74 696d weroff) (cpu_tim 270: 6520 342e 3731 3030 3838 3834 3829 2028 e 4.710088848) ( 280: 6f6e 6c69 6e65 5f76 6370 7573 2031 2920 online_vcpus 1) 290: 2875 705f 7469 6d65 2032 392e 3131 3037 (up_time 29.1107 2a0: 3332 3037 3836 2920 2873 7461 7274 5f74 320786) (start_t 2b0: 696d 6520 3131 3539 3439 3932 3135 2e30 ime 1159499215.0 2c0: 3629 2028 7374 6f72 655f 6d66 6e20 3533 6) (store_mfn 53 2d0: 3130 3930 2920 2863 6f6e 736f 6c65 5f6d 1090) (console_m 2e0: 666e 2035 3331 3038 3929 2900 0001 fn 531089)). 2f0: 0001 0002 300: 310: f100 1000 320: 00 ... ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel][PATCH][RFC] Task: support huge page RE:[Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11460
Hi Magnus, From: Magnus Damm [mailto:[EMAIL PROTECTED] Sent: 2006年9月28日 21:12 To: Isaku Yamahata Cc: Xu, Anthony; Tristan Gingold; Alex Williamson; xen-ia64-devel@lists.xensource.com Subject: Re: [Xen-ia64-devel][PATCH][RFC] Task: support huge page RE:[Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11460 Large contiguous memory chunks - ouch. I know that Mel Gorman has been working on the Linux side of hugetlbfs with both antifragmentation and defragmentation. I think dedicated pools for large pages is a simle and good solution - trying to allocate large contiguous chunks of memory from a global memory pool without any support for defragmentation/migration is begging for trouble IMO. Implementing defragmentation is necessary, But it is not easy, as I know there several defragmentation algorithms discussed about hugetlbfs in linux kernel, until now, there is no one algorithm overwhelming. Maybe we can postpone this. Creating/Destroying domain frequently in XEN/IA64 may be not the case. It may be not important as it seems in XEN/IA64. I agree with you that supporting tlb insert with large page size and hugetlbfs would be a big gain. Yes the theory (and your numbers) all say that larger page sizes are good for performance. I think page table code is fun too, so I must admit that I'm a bit tempted. That's great! I'm not sure of the current status of Xen and N-order allocations, but I have a feeling that it is possible to keep the complexity low and still get some performance improvement by limiting the scope to some kind of dedicated huge page pool and a PV interface to hugetlbfs. Dedicated huge page pool may be a good idea for linux kernel. As for XEN/IA64, Xen doesn't know which address space should be contiguous for guest. From guest view, all the physical address is contiguous. The simplest method is to allocate enough big contiguous memory chunks for guest, For instance, in RHEL4-U2, this biggest tlb is 256M, then XEN allocate 256M contiguous memory chunks for guest. Or, just make sure that each dom0 is loaded into contiguous memory and do copy instead of flipping. In my mind, we need do below things (there may be more) if we want to support huge page. 1. Add an option order in configure file vtiexample.vti. if order=0, XEN/IA64 allocate 16K contiguous memory for domain, if order=1, allocate 32K, and so on. Thus user can chose page size for domain. A fall back path should be implemented in case that large page allocation fails. Or do you propose introducing new page allocator with very large chunk? With order option, page fragmentation should be taken care of. I think the best soltion would be to divide memory into different pools at boot time to avoid fragmentation problems. If large page allocation fails when creating a domain - then just return error telling the user that he has misconfigured his system. That is exactly what I'm thinking, Again, we can use buddy system to allocate huge page first, Then turn to dedicated pools method. In fact, these two parts can be developed in the same time 6. Ballon driver may need to be modified to increase or decrease domain memory by page size not 16K. Magnus, would you like to take this task? I'm currently busy with VT-extension work for kexec on x86 and x86_64, and on top of that I feel that it may be too complex for me. Especially since I have very little ia64 experience. I'm interested in doing the x86 side of it though if someone else drives the ia64 bit. That's Ok. As Ian talked in Xen summit, XEN/IA32 may need to support huge page, there are also huge pages in IA32, 4M and 2M. kernel mapping with large page size. * page fragmentation should be addressed. Both 16KB and 16MB page should be able to co-exist in a same domain. - Allocating large contiguous region might fail. So fall back path should be implemented. - domain should be able to have pages with both page size (16KB and 16MB) for smooth code merge. probably a new bit of the p2m entry, something like _PAGE_HPAGE, would be introduce to distinguish large page from normal page. On PV it may be possible to setup two address spaces for the kernel - one using huge pages and another one with smaller pages. Then the area with huge pages is used for read-only operation and other activites such as page flipping can be performed in the small page area. I have below concern, For instance, there are 4G in PV, 2G for huge pages, and 2G for smaller pages. That means PV can only use 2G to huge pages, and VBD/VNIF can only use 2G, even though there are 4G. * paravirtualized driver(VBD/VNIF) This is a really issue. For first prototype it is reasonable to not support page flipping resorting grant table memory copy. There are two kinds of page flipping, page mapping and page transfer. I guess page mapping should be supported somehow assuming only dom0 (or driver domain) maps. We should measure
Re: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11635
Hi Yongkang, This issue was found in Cset#10875. But the rate of reproducing is low. --- [EMAIL PROTECTED] xen]# xm info (snip) xen_changeset : Mon Jul 31 15:14:47 2006 -0600 10875:00dd5eb7adc1 (snip) [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1004 3 r- 5598.9 [EMAIL PROTECTED] xen]# (xm create rhel4.test1 );xm create rhel4.test2 [1] 7801 [EMAIL PROTECTED] xen]# Using config file rhel4.test1. Using config file rhel4.test2. Started domain rhel4.DomU-1 Started domain rhel4.DomU-2 [1]+ Donexm create rhel4.test2 [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1004 3 r- 5706.0 Zombie-rhel4.DomU-15 1024 1 cd 5.7 Zombie-rhel4.DomU-26 1024 1 cd 5.6 rhel4.DomU-1 8 1024 1 r- 0.0 [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1004 3 r- 5731.6 Zombie-rhel4.DomU-15 1024 1 cd 5.7 Zombie-rhel4.DomU-18 1024 1 cd 0.1 Zombie-rhel4.DomU-26 1024 1 cd 5.6 [EMAIL PROTECTED] xen]# xend restart [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1004 3 r- 5812.2 [EMAIL PROTECTED] xen]# xm create rhel4.test2 Using config file rhel4.test2. Started domain rhel4.DomU-2 [EMAIL PROTECTED] xen]# xm list Error: Device 0 not connected [EMAIL PROTECTED] xen]# xm list Error: Device 769 not connected [EMAIL PROTECTED] xen]# xend restart [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1004 3 r- 5973.6 [EMAIL PROTECTED] xen]# xm create rhel4.test1 Using config file rhel4.test1. Started domain rhel4.DomU-1 [EMAIL PROTECTED] xen]# xm list Error: Device 0 not connected [EMAIL PROTECTED] xen]# xm list Error: Device 769 not connected [EMAIL PROTECTED] xen]# xend restart [EMAIL PROTECTED] xen]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1004 3 r- 6090.2 --- Best Regards, Katase - Original Message - From: You, Yongkang [EMAIL PROTECTED] To: Tristan Gingold [EMAIL PROTECTED]; xen-ia64-devel@lists.xensource.com Sent: Thursday, September 28, 2006 3:43 PM Subject: RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11635 Hi Tristan, We found it for a long time. This issue also can be easily reproduced in latest changeset 11635. I guess it might be the common issue with IA32 side. But IA32 xenU didn't meet the problem with the same operations. Sorry for the wrong subject 11609, I send out yesterday. It should be 11635. Best Regards, Yongkang (Kangkang) 永康 -Original Message- From: Tristan Gingold [mailto:[EMAIL PROTECTED] Sent: 2006年9月28日 14:41 To: You, Yongkang; xen-ia64-devel@lists.xensource.com Subject: Re: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11609 Le Mercredi 27 Septembre 2006 12:05, You, Yongkang a écrit : The 2 XenU coexisting testing is easy to fail. I try 3 times to create 2 XenU domains at the same time (xm create config1 ) ; xm create config2 . The first 2 times are pass. But the 3rd time. XenU creating failed. After that, xm list can not list domains and will report Error: Device 768 not connected Use xend stop and xend start could recover xm list. But if try to create 2 XenU domains again, there will be 4 Zombie-Domains in xm list. It is same with the bug http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=747 Hi, Is it a recent bug ? According to the bugzilla date it seems so (Aug 29). Could you try to find the changeset from which this bug appears ? Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel][PATCH][RFC] Task: support huge page RE: [Xen-ia64-devel] Xen/IA64 Healthiness Report -Cset#11460
Hi Anthony. On Fri, Sep 29, 2006 at 10:28:36AM +0800, Xu, Anthony wrote: Page allocator is still based on 16K page, otherwise will impact small memory allocation, such as xmalloc, and there are many places need to be modified. So I think the first step is Page allocator still use 16K page, but we allocate huge page for domain. As for allocation failure, the first step is, if allocation fails, creating domain fails, The next step is to defragmentation. For the first step, it sounds reasonable to not care about allocation failure. I think that page fragmentation should be addressed as a eventual goal. It might have an impact on its design. 4. Per_LP_VHPT may need to be modified to support huge page. Do you mean hash collision? I don't mean hash collision. Per_LP_VHPT is long format VHPT, it can support huge page essentially, but many code about VHPT assume page size is 16K, so itir.ps is always 16K in some code sequence. Understood. * Presumably there are two goals - Support one large page size(e.g. 16MB) to map kernel. - Support hugetlbfs whose page size might be different from 16MB. I.e. support three page sizes, normal page size 16KB, kernel mapping page size 16MB and hugetlbfs page size 256MB. I think hugetlbfs support can be addressed specialized way. Kernel is using 16M identity mapping, and rr7.ps=16M, so if Xen allocate 16M contiguous chunks for domain, then Xen can set machine rr7.ps 16M instead of 16K, then all VHPT entries for region 7 in Per_LP_VHPT is 16M page size. I'm using rhel4-u2 as guest, by default, rhel4-u2 set rr4.ps=256M. For latest kernel who supports hugetlbfs, the biggest page size is 4G. The goals from me is supporting 256M, if we can do that, and then supporting huger tlb like 1G or 4G is trivial. :-) I wanted to say that a implementation for hugetlbfs may be different from a implementation for large page to map kernel. So we should describe not only page-size, but also its purpose. I re-summarize the goals. What do you think? - Support 16MB large page size to map kernel area. Although Linux can be configured to use 64MB page size to map kernel, we won't support 64MB page size. (at least for first prototype) It would be nice to support both kernel mapping page size, 16MB and 64MB, but it might be addressed by second phase. A domain uses only one of them. Potentially different domains may use different kernel mapping page size. e.g. domainA uses 16MB to map kernel, domainB uses 64MB to map kernel. - Support hugetlbfs with 256MB page size. If possible, it would be nice to support 1GB page size or 4GB page size. The huge page size is determined by Linux boot time options and only single page size from 256MB, 1GB, 4GB is used by hugetlbfs of a domain. Potentially different domains may use different huge page size. e.g. domainA uses 256MB huge page size, domainB uses 1GB huge page size. For first prototype, it is reasonable to support only 256MB huge page size. - page fragmentation should be addressed. This isn't addressed at the first step. When large contiguous page allocation fails, fall back path is executed with normal page size, 16KB, possibly at degraded performance. Or try a sort of defragmentation to create a large contiguous memory. hugetlbfs * Some specialized path can be implemented to support hugetlbfs. - For domU paravirtualize hugetlbfs for domU. Hook to alloc_fresh_huge_page() in Linux. Then xen/ia64 is aware of large pages. Probably a new flag of the p2m entry, or other data structure might be introduced. For xenLinux, the region number, RGN_HPAGE can be used to check before entering hugetlbfs specialized path. That's good, but first Xen need to allocate contiguous chunks for domU - For domVTI Can the use of hugetlbfs be detected somehow? In domVTI side, Xen don't know hugetlbfs, but Xen can capture guest accessing rr, If a new preferred page size is set ( rr.ps),( preferred page size means this page size is used mostly in this region). If Xen can set the same preferred page size into machine rr.ps, that's great, most tlb miss can be handled by assembly code, that means it can be found in long format VHPT, otherwise Xen need to lookup VTLB in C code Hutetlbfs page size is Linux boot time option. So I think it might be acceptable to require domain configuration option. You already added order options in your proto type in fact. thanks. -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel