Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
Today i've built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 via Michael's http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e. # Xen patches ApplyPatch xen.next-2.6.37.patch # ApplyPatch xen.upstream.core.patch ApplyPatch xen.pcifront.fixes.patch # ApplyPatch xen.pvhvm.fixes.patch as a result i got a kernel wich runs pretty stable NFS client at Xen 4.0.1 F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64). I was able several times copied from NFS folder F14's ISO image (3.2 GB) to DomU and scp'ed it back and didn't get any kernel crashing on DomU. On Ubuntu 10.04 this kernel may be built as rc1&git8 patched via xen.next-2.6.37.patch xen.pcifront.fixes.patch All required upstream patches may be taken (as well as 2 above) from link http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 I believe as soon as xen.pcifront.fixes.patch will be accepted by upstream NFS client issue on F14 will be gone Boris. --- On Mon, 11/15/10, Boris Derzhavets wrote: From: Boris Derzhavets Subject: Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel To: "M A Young" Cc: xen@lists.fedoraproject.org Date: Monday, November 15, 2010, 1:13 PM I've tested the most recent kernel from Michael Young http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64(rc1.git8 + xen-next.patch) as Dom0 kernel under Xen 4.0.1 on top of F14. Attempt to work with NFS remote folder results kernel crash again - unable to handle paging request and stack trace. Kernel works OK only as vanilla. Boris. P.S. Same issue is in place for PV DomU running kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 -Inline Attachment Follows- -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
I've tested the most recent kernel from Michael Young http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64(rc1.git8 + xen-next.patch) as Dom0 kernel under Xen 4.0.1 on top of F14. Attempt to work with NFS remote folder results kernel crash again - unable to handle paging request and stack trace. Kernel works OK only as vanilla. Boris. P.S. Same issue is in place for PV DomU running kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
On Sat, Nov 13, 2010 at 6:41 AM, M A Young wrote: > There is another kernel to play with (2.6.37-0.1.rc1.git8.xendom0.fc15) > at http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 . > This has patches from xen/next-2.6.37 , which includes the gntdev module, > so it should in theory be possible to use a userspace blkback. I still get a dom0 crash, whereas 2.6.32.25-172.xendom0.fc12.i686.PAE works fine on the same hardware with the same command line. jerry __ ___ ____ _ __ __ _ _ _ \ \/ /___ _ __ | || | / _ \ / | / /_ / _| ___/ | || | \ // _ \ '_ \ | || |_| | | || |__| '_ \ | |_ / __| | || |_ / \ __/ | | | |__ _| |_| || |__| (_) || _| (__| |__ _| /_/\_\___|_| |_||_|(_)___(_)_| \___(_)_| \___|_| |_| (XEN) Xen version 4.0.1 (mockbuild@(none)) (gcc version 4.5.1 20100924 (Red Hat 4.5.1-4) (GCC) ) Tue Oct 12 22:05:55 UTC 2010 (XEN) Latest ChangeSet: unavailable (XEN) Console output is synchronous. (XEN) Bootloader: GNU GRUB 0.97 (XEN) Command line: console=com1 com1=38400,8n1 console=com1 sync_console loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 2 seconds (XEN) Disc information: (XEN) Found 4 MBR signatures (XEN) Found 4 EDD information structures (XEN) Xen-e820 RAM map: (XEN) - 000a (usable) (XEN) 0010 - 7ffc (usable) (XEN) 7ffc - 7ffcfc00 (ACPI data) (XEN) 7ffcfc00 - 7000 (reserved) (XEN) fec0 - fec9 (reserved) (XEN) fed2 - fed8 (reserved) (XEN) fee0 - fee1 (reserved) (XEN) ffb0 - 0001 (reserved) (XEN) System RAM: 2047MB (2096512kB) (XEN) ACPI: RSDP 000FDC40, 0014 (r0 DELL ) (XEN) ACPI: RSDT 000FDC54, 0030 (r1 DELL PE700 1 MSFT 10A) (XEN) ACPI: FACP 000FDC84, 0074 (r1 DELL PE700 1 MSFT 10A) (XEN) ACPI: DSDT 7FFC, 1821 (r1 DELLPE7xx 1 MSFT 10E) (XEN) ACPI: FACS 7FFCFC00, 0040 (XEN) ACPI: APIC 000FDCF8, 0074 (r1 DELL PE700 1 MSFT 10A) (XEN) ACPI: SPCR 000FDD6C, 0050 (r1 DELL PE700 1 MSFT 10A) (XEN) No NUMA configuration found (XEN) Faking a node at -7ffc (XEN) Xen heap: 8MB (8856kB) (XEN) Domain heap initialised (XEN) found SMP MP-table at 000fe710 (XEN) DMI 2.3 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x808 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[804,0], pm1x_evt[800,0] (XEN) ACPI: wakeup_vec[7ffcfc0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee0 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) Processor #0 15:3 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] disabled) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x01] address[0xfec0] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 1, version 32, address 0xfec0, GSI 0-23 (XEN) ACPI: IOAPIC (id[0x02] address[0xfec1] gsi_base[24]) (XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec1, GSI 24-47 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 2793.137 MHz processor. (XEN) CPU: Hyper-Threading is disabled (XEN) CPU0: Intel Extended MCE MSRs (12) available (XEN) Intel machine check reporting enabled (XEN) I/O virtualisation disabled (XEN) Total of 1 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) Platform timer is 3.579MHz ACPI PM Timer (XEN) Allocated console ring of 16 KiB. (XEN) Brought up 1 CPUs (XEN) CPUIDLE: disabled due to no HPET. Force enable with 'cpuidle'. (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 32-bit, PAE, lsb (XEN) Dom0 kernel: 32-bit, PAE, lsb, paddr 0x40 -> 0x15ac000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 3800->3c00 (468836 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: c040->c15ac000 (XEN) Init. ramdisk: c15ac000->c327da00 (XEN) Phys-Mach map: c327e000->c3457d90 (XEN) Start info:c3458000->c345847c (XEN) Page tables: c3459000->c347a000 (XEN) Boot stack:c347a000->c347b000 (XEN) TOTAL: c000->c380 (XEN) ENTRY ADDRESS: c0b05000 (XEN) Dom0 has maximum 1 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) trace.c:89:d32767 calc_tinfo_firs
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
It should be correct patch :- - %define nopatches 1 %if %{nopatches}%{using_upstream_branch} %define listnewconfig_fail 0 %else %define listnewconfig_fail 1 %endif --- Boris. --- On Sat, 11/13/10, M A Young wrote: From: M A Young Subject: Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel To: "Boris Derzhavets" Cc: xen@lists.fedoraproject.org Date: Saturday, November 13, 2010, 2:49 PM On Sat, 13 Nov 2010, Boris Derzhavets wrote: > If i just want to build fedora kernel for rc1 & git9 script fails due to > listnewconfig_fail = 1 That means a new configuration option has been added somewhere. If you look through the output of the failed build it will tell you what the new added option is. So an alternative to your workaround is to set this new option in the config files. Michael Young -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
On Sat, 13 Nov 2010, Boris Derzhavets wrote: > If i just want to build fedora kernel for rc1 & git9 script fails due to > listnewconfig_fail = 1 That means a new configuration option has been added somewhere. If you look through the output of the failed build it will tell you what the new added option is. So an alternative to your workaround is to set this new option in the config files. Michael Young -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
If i just want to build fedora kernel for rc1 & git9 script fails due to listnewconfig_fail = 1 Simple hack :- # Should make listnewconfig fail if there's config options # printed out? %if %{nopatches}%{using_upstream_branch} %define listnewconfig_fail 0 %else %define listnewconfig_fail 0 %endif I would guess, that it shouldn't happen in general. Boris. --- On Sat, 11/13/10, M A Young wrote: From: M A Young Subject: Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel To: xen@lists.fedoraproject.org Date: Saturday, November 13, 2010, 7:41 AM There is another kernel to play with (2.6.37-0.1.rc1.git8.xendom0.fc15) at http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 . This has patches from xen/next-2.6.37 , which includes the gntdev module, so it should in theory be possible to use a userspace blkback. My testing with a no network or disk guest are as successful as I could expect them to be. Michael Young -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
There is another kernel to play with (2.6.37-0.1.rc1.git8.xendom0.fc15) at http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 . This has patches from xen/next-2.6.37 , which includes the gntdev module, so it should in theory be possible to use a userspace blkback. My testing with a no network or disk guest are as successful as I could expect them to be. Michael Young -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
I've tried to adjust kernel.spec from kernel-2.6.37-0.1.rc0.git13.xendom0.fc15.src.rpm to rcrevision = 1, gitrevision =5 . # The rc snapshot level %define rcrev 1 # The git snapshot level %define gitrev 5 . Files patch-2.6.37-rc1.bz2, patch-2.6.37-rc1-git5.bz2 have been copied to SOURCES I get an error when attempt to build :- # rpmbuild -bb ./kernel.spec ... Patch161: linux-2.6-i386-nx-emulation.patch + case "$patch" in + patch -p1 -F1 -s + ApplyPatch linux-2.6-32bit-mmap-exec-randomization.patch + local patch=linux-2.6-32bit-mmap-exec-randomization.patch + shift + '[' '!' -f /root/rpmbuild/SOURCES/linux-2.6-32bit-mmap-exec-randomization.patch ']' Patch160: linux-2.6-32bit-mmap-exec-randomization.patch + case "$patch" in + patch -p1 -F1 -s 1 out of 5 hunks FAILED -- saving rejects to file mm/mmap.c.rej error: Bad exit status from /var/tmp/rpm-tmp.FmT4Fy (%prep) Boris. P.S. I've also tried to adjust kernel.spec from kernel-2.6.37-0.1.rc1.git0.xendom0.fc15.src.rpm with results even worse then above:- Patch32: linux-2.6-utrace-ptrace.patch failed to apply. --- On Sat, 11/6/10, M A Young wrote: From: M A Young Subject: Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel To: "Boris Derzhavets" Cc: xen@lists.fedoraproject.org, "xen-users" Date: Saturday, November 6, 2010, 9:29 AM On Sat, 6 Nov 2010, Boris Derzhavets wrote: > ... > [ 42.140012] Call Trace: > [ 42.140012] [] ? print_irqtrace_events+0xa0/0xa4 > [ 42.140012] [] __might_sleep+0x103/0x108 > [ 42.140012] [] mutex_lock_nested+0x25/0x43 > [ 42.140012] [] percpu_counter_destroy+0x3c/0x66 > [ 42.140012] [] percpu_counter_fixup_free+0x16/0x32 > [ 42.140012] [] debug_object_fixup+0x1e/0x2b > [ 42.140012] [] debug_check_no_obj_freed+0xce/0x18a > ... This is probably a result of a percpu_counter bug in an IPv6 patch. As a temporary measure you can disable IPv6 (if you don't need it), eg. as suggested at http://fedorasolved.org/network-solutions/disable-ipv6 It was patched after rc1 http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6-stable.git;a=commit;h=41bb78b4b9adb21cf2c395b6b880aaae99c788b7 so should be fixed when I next do a build of 2.6.37 . Michael Young -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
On Sat, 6 Nov 2010, Boris Derzhavets wrote: ... [ 42.140012] Call Trace: [ 42.140012] [] ? print_irqtrace_events+0xa0/0xa4 [ 42.140012] [] __might_sleep+0x103/0x108 [ 42.140012] [] mutex_lock_nested+0x25/0x43 [ 42.140012] [] percpu_counter_destroy+0x3c/0x66 [ 42.140012] [] percpu_counter_fixup_free+0x16/0x32 [ 42.140012] [] debug_object_fixup+0x1e/0x2b [ 42.140012] [] debug_check_no_obj_freed+0xce/0x18a ... This is probably a result of a percpu_counter bug in an IPv6 patch. As a temporary measure you can disable IPv6 (if you don't need it), eg. as suggested at http://fedorasolved.org/network-solutions/disable-ipv6 It was patched after rc1 http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6-stable.git;a=commit;h=41bb78b4b9adb21cf2c395b6b880aaae99c788b7 so should be fixed when I next do a build of 2.6.37 . Michael Young-- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen
Re: [Fedora-xen] 2.6.37-rc1 dom0 kernel
Dmesg log is attached :- [ 42.140012] WARNING: at lib/debugobjects.c:259 debug_print_object+0x5b/0x63() [ 42.140012] Hardware name: System Product Name [ 42.140012] ODEBUG: free active (active state 0) object type: percpu_counter [ 42.140012] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat deflate zlib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key sunrpc bridge stp llc ipv6 evtchn xenfs uinput snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel r8169 mii snd_hda_codec snd_hwdep shpchp snd_seq snd_seq_device snd_pcm microcode snd_timer iTCO_wdt i2c_i801 asus_atk0110 snd iTCO_vendor_support soundcore snd_page_alloc pata_acpi firewire_ohci firewire_core ata_generic crc_itu_t pata_jmicron radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core [last unloaded: scsi_wait_scan] [ 42.140012] Pid: 53, comm: kworker/u:6 Not tainted 2.6.37-0.1.rc1.git0.xendom0.fc14.x86_64 #1 [ 42.140012] Call Trace: [ 42.140012] [] warn_slowpath_common+0x85/0x9d [ 42.140012] [] warn_slowpath_fmt+0x46/0x48 [ 42.140012] [] debug_print_object+0x5b/0x63 [ 42.140012] [] debug_check_no_obj_freed+0x99/0x18a [ 42.140012] [] ? arch_local_save_flags+0xb/0xd [ 42.140012] [] ? debug_check_no_locks_freed+0x11c/0x138 [ 42.140012] [] ? net_free+0x2c/0x31 [ 42.140012] [] kmem_cache_free+0x6f/0x10b [ 42.140012] [] net_free+0x2c/0x31 [ 42.140012] [] cleanup_net+0x178/0x197 [ 42.140012] [] process_one_work+0x1f4/0x361 [ 42.140012] [] ? process_one_work+0x160/0x361 [ 42.140012] [] ? cleanup_net+0x0/0x197 [ 42.140012] [] worker_thread+0x104/0x1a4 [ 42.140012] [] ? worker_thread+0x0/0x1a4 [ 42.140012] [] kthread+0xa0/0xa8 [ 42.140012] [] ? trace_hardirqs_on_caller+0x10b/0x12f [ 42.140012] [] kernel_thread_helper+0x4/0x10 [ 42.140012] [] ? restore_args+0x0/0x30 [ 42.140012] [] ? kernel_thread_helper+0x0/0x10 [ 42.140012] ---[ end trace c2d2451e1a620a4f ]--- [ 42.140012] BUG: sleeping function called from invalid context at kernel/mutex.c:278 [ 42.140012] in_atomic(): 0, irqs_disabled(): 1, pid: 53, name: kworker/u:6 [ 42.140012] 2 locks held by kworker/u:6/53: [ 42.140012] #0: (netns){+.+.+.}, at: [] process_one_work+0x160/0x361 [ 42.140012] #1: (net_cleanup_work){+.+.+.}, at: [] process_one_work+0x160/0x361 [ 42.140012] irq event stamp: 7666 [ 42.140012] hardirqs last enabled at (7665): [] kfree+0x11f/0x136 [ 42.140012] hardirqs last disabled at (7666): [] kmem_cache_free+0x38/0x10b [ 42.140012] softirqs last enabled at (7620): [] netlink_release+0x1f3/0x208 [ 42.140012] softirqs last disabled at (7618): [] netlink_release+0x1db/0x208 [ 42.140012] Pid: 53, comm: kworker/u:6 Tainted: G W 2.6.37-0.1.rc1.git0.xendom0.fc14.x86_64 #1 [ 42.140012] Call Trace: [ 42.140012] [] ? print_irqtrace_events+0xa0/0xa4 [ 42.140012] [] __might_sleep+0x103/0x108 [ 42.140012] [] mutex_lock_nested+0x25/0x43 [ 42.140012] [] percpu_counter_destroy+0x3c/0x66 [ 42.140012] [] percpu_counter_fixup_free+0x16/0x32 [ 42.140012] [] debug_object_fixup+0x1e/0x2b [ 42.140012] [] debug_check_no_obj_freed+0xce/0x18a [ 42.140012] [] ? debug_check_no_locks_freed+0x11c/0x138 [ 42.140012] [] ? net_free+0x2c/0x31 [ 42.140012] [] kmem_cache_free+0x6f/0x10b [ 42.140012] [] net_free+0x2c/0x31 [ 42.140012] [] cleanup_net+0x178/0x197 [ 42.140012] [] process_one_work+0x1f4/0x361 [ 42.140012] [] ? process_one_work+0x160/0x361 [ 42.140012] [] ? cleanup_net+0x0/0x197 [ 42.140012] [] worker_thread+0x104/0x1a4 [ 42.140012] [] ? worker_thread+0x0/0x1a4 [ 42.140012] [] kthread+0xa0/0xa8 [ 42.140012] [] ? trace_hardirqs_on_caller+0x10b/0x12f [ 42.140012] [] kernel_thread_helper+0x4/0x10 [ 42.140012] [] ? restore_args+0x0/0x30 [ 42.140012] [] ? kernel_thread_helper+0x0/0x10 Boris --- On Fri, 11/5/10, M A Young wrote: From: M A Young Subject: [Fedora-xen] 2.6.37-rc1 dom0 kernel To: xen@lists.fedoraproject.org Date: Friday, November 5, 2010, 7:32 PM I have built another test kernel (2.6.37-0.1.rc1.git0.xendom0.fc15) to play with at http://koji.fedoraproject.org/koji/taskinfo?taskID=2581020 . As with the previous kernel this is entirely based on the upstream kernel and Fedora kernel patches, and is probably close to what the Fedora 15 kernel will look like soon. I haven't tested it yet. Michael Young -- xen mailing list xen@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/xen dmesg.log.gz Description: GNU Zip compressed data -- xen mailing list xen@li