RE: [XenPPC] systemsim-gpul problems
Not sure this sheds any light on the timing problem but it looks like Xen is in it's idle_loop() during the time things are running unexpectedly slow. Stuart -Original Message- From: Hollis Blanchard [mailto:[EMAIL PROTECTED] Sent: Friday, January 05, 2007 1:06 PM To: Yoder Stuart-B08248 Cc: Jimi Xenidis; xen-ppc-devel@lists.xensource.com Subject: RE: [XenPPC] systemsim-gpul problems On Fri, 2007-01-05 at 09:54 -0700, Yoder Stuart-B08248 wrote: 1449668513: (1449502549): IPv4 over IPv4 tunneling driver 1450457186: (1450290832): TCP bic registered 1450527304: (1450360932): NET: Registered protocol family 1 1450557364: (1450390979): NET: Registered protocol family 17 146000: [0]: (PC:0x0042FA80) : 7997.4 Kilo-In... 148000: [0]: (PC:0x0042FA80) : .3 Kilo-In... 15: [0]: (PC:0x004224B0) : .3 Kilo-In... This is interesting because it seems to indicate that we're spending lots of time in Xen (that's a Xen address). nm xen-syms | sort should help you figure out what function that is. -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
RE: [XenPPC] systemsim-gpul problems
-Original Message- From: Jimi Xenidis [mailto:[EMAIL PROTECTED] Sent: Thursday, January 04, 2007 7:54 PM To: Hollis Blanchard Cc: Yoder Stuart-B08248; xen-ppc-devel@lists.xensource.com Subject: Re: [XenPPC] systemsim-gpul problems On Jan 4, 2007, at 7:19 PM, Hollis Blanchard wrote: On Thu, 2007-01-04 at 16:48 -0700, Yoder Stuart-B08248 wrote: Whad'ya know...if you wait long enough it works! So simulator performance is acceptable until mid-way through dom0 boot? It would be good to figure out the source of the slowdown. the last time we had this problem it was because Linux takes to long figuring out instructions-per-jiffy and then the result is pain. can try the workaround described in: http://lists.xensource.com/archives/html/xen-ppc-devel/2006-03/ msg00119.html We may have a systemsym regression here, because this was no longer necessary. -JX I tried the lpj= option and it didn't seem to affect the wall clock time the simluation takes. Here is an observation-- On the changset from 11/21 that performed more quickly here is an excerpt of the log: [snip] 1449091137: (1249159347): IPv4 over IPv4 tunneling driver 1449879806: (1249947628): TCP bic registered 1449948355: (1250016163): NET: Registered protocol family 1 1449978413: (1250046208): NET: Registered protocol family 17 2310238380: (1251708119): Bridge firewalling registered 2310251180: (1251720893): Ebtables v2.0 registered 2670267306: (1252438049): ebt_ulog: not logging via ulog... 2670949205: (1253119878): 802.1Q VLAN Support v1.8 Ben G... 2670966817: (1253137486): All bugs added by David S. Mil... [snip] After the systemsim-gpul fix went in on 12/11 here is the same excerpt of the log [snip 1449668513: (1449502549): IPv4 over IPv4 tunneling driver 1450457186: (1450290832): TCP bic registered 1450527304: (1450360932): NET: Registered protocol family 1 1450557364: (1450390979): NET: Registered protocol family 17 146000: [0]: (PC:0x0042FA80) : 7997.4 Kilo-In... 148000: [0]: (PC:0x0042FA80) : .3 Kilo-In... 15: [0]: (PC:0x004224B0) : .3 Kilo-In... 152000: [0]: (PC:0x004224AC) : .3 Kilo-In... 154000: [0]: (PC:0x0042FA84) : .2 Kilo-In... 156000: [0]: (PC:0x004224B8) : .3 Kilo-In... 158000: [0]: (PC:0x0042FA74) : .3 Kilo-In... 16: [0]: (PC:0x0042FA80) : 2500.1 Kilo-In... 162000: [0]: (PC:0x004224B0) : .3 Kilo-In... 164000: [0]: (PC:0x0042FA80) : .3 Kilo-In... 166000: [0]: (PC:0x004224A8) : .3 Kilo-In... 168000: [0]: (PC:0x0042FA84) : .3 Kilo-In... 17: [0]: (PC:0x0042FA78) : .3 Kilo-In... 172000: [0]: (PC:0x0042FA78) : .3 Kilo-In... 174000: [0]: (PC:0x004224B8) : .3 Kilo-In... 176000: [0]: (PC:0x004224B0) : 2500.1 Kilo-In... 178000: [0]: (PC:0x0042FA80) : .3 Kilo-In... 18: [0]: (PC:0x004224A8) : .3 Kilo-In... 182000: [0]: (PC:0x004224B0) : .3 Kilo-In... 184000: [0]: (PC:0x0042FA74) : .3 Kilo-In... 186000: [0]: (PC:0x0042FA74) : .3 Kilo-In... 188000: [0]: (PC:0x0042FA74) : .3 Kilo-In... 19: [0]: (PC:0x004224A8) : .3 Kilo-In... 192000: [0]: (PC:0x004224AC) : 2500.1 Kilo-In... 194000: [0]: (PC:0x0042FA78) : .3 Kilo-In... 196000: [0]: (PC:0x0042FA80) : .3 Kilo-In... 198000: [0]: (PC:0x0042FA74) : .3 Kilo-In... 20: [0]: (PC:0x0042FA84) : .3 Kilo-In... 202000: [0]: (PC:0x004224B0) : .3 Kilo-In... 204000: [0]: (PC:0x0042FA78) : .3 Kilo-In... 206000: [0]: (PC:0x004224B8) : .3 Kilo-In... 208000: [0]: (PC:0x0042FA78) : .3 Kilo-In... 21: [0]: (PC:0x004224B0) : .3 Kilo-In... 212000: [0]: (PC:0x004224B0) : 2500.1 Kilo-In... 214000: [0]: (PC:0x0042FA80) : .3 Kilo-In... 216000: [0]: (PC:0x004224AC) : .3 Kilo-In... 218000: [0]: (PC:0x0042FA78) : .3 Kilo-In... 22: [0]: (PC:0x0042FA74) : .3 Kilo-In... 222000: [0]: (PC:0x004224B8) : .3 Kilo-In... 224000: [0]: (PC:0x004224AC) : .3 Kilo-In... 226000: [0]: (PC:0x0042FA78) : 2500.1 Kilo-In... 228000: [0]: (PC:0x004224B0) : .3 Kilo-In... 23: [0]: (PC:0x004224B8) : .3 Kilo-In... 2311211643: (2311063340): Bridge firewalling registered 231122: (2311076116): Ebtables v2.0 registered 232000: [0]: (PC:0x004224B8) : .3 Kilo-In... 234000: [0]: (PC:0x0042FA84) : .3 Kilo-In... 236000: [0]: (PC:0x004224B0) : .3 Kilo-In... 238000: [0]: (PC
RE: [XenPPC] systemsim-gpul problems
On Fri, 2007-01-05 at 09:54 -0700, Yoder Stuart-B08248 wrote: 1449668513: (1449502549): IPv4 over IPv4 tunneling driver 1450457186: (1450290832): TCP bic registered 1450527304: (1450360932): NET: Registered protocol family 1 1450557364: (1450390979): NET: Registered protocol family 17 146000: [0]: (PC:0x0042FA80) : 7997.4 Kilo-In... 148000: [0]: (PC:0x0042FA80) : .3 Kilo-In... 15: [0]: (PC:0x004224B0) : .3 Kilo-In... This is interesting because it seems to indicate that we're spending lots of time in Xen (that's a Xen address). nm xen-syms | sort should help you figure out what function that is. -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
RE: [XenPPC] systemsim-gpul problems
Whad'ya know...if you wait long enough it works! Thanks Mark. From: Mark F Mergen [mailto:[EMAIL PROTECTED] Sent: Thursday, January 04, 2007 4:30 PM To: Yoder Stuart-B08248; xen-ppc-devel@lists.xensource.com Subject: Fw: [XenPPC] systemsim-gpul problems Sorry, I should have read your note more carefully. Yes, my latest builds do boot all the way into Linux, but the timing is drastically altered from the past. There are places in the boot sequence (and I think what you mention: after protocol family 17 is one of them) where you must wait a VERY long time for something to happen. I can't tell you exactly how long, but it took me a while and accidental inattention to what was happening before I discovered this. I have not had time to work on why this is so. Also, if you do get all the way to a Linux command prompt, you may conclude it is not responding to commands. Just type a command that you think should work (such as pwd), hit Enter, and then WAIT A LONG TIME. If your experience turns out to be like mine, eventually the characters you typed will be echoed and the command will be executed with proper output, however slowly. Obviously, more debugging is needed. Maybe it's something simple, like some simulator option for how real time is reflected to the simulated machine. Maybe you will be rewarded by a little (or a LOT of) patience in your next try. Mark Mergen - Forwarded by Mark F Mergen/Watson/IBM on 01/04/2007 05:15 PM - Mark F Mergen/Watson/IBM 01/04/2007 05:12 PM To [EMAIL PROTECTED] cc Yoder Stuart-B08248 [EMAIL PROTECTED], xen-ppc-devel@lists.xensource.com Subject Re: [XenPPC] systemsim-gpul problemsLink Notes://D01MLC04/852564F1000C2D56/C57E6BC8EC0391E7852571CA007FA25B/9BD9 00E03249E9608525725900705623 I pull only from xenppc-unstable.hg and linux-ppc-2.6.hg. I'm up to date with all latest that was pushed to them, and this combo runs on systemsim-gpul. There was a bug(s?) that caused symptoms like you mention, but they were fixed by Amos Waterland about 2nd week in December, and pushed to aformentioned repositories by the maintainers. I can't quote or vouch for specific changesets. Mark Mergen [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/04/2007 03:11 PM Please respond to [EMAIL PROTECTED] To Yoder Stuart-B08248 [EMAIL PROTECTED] cc xen-ppc-devel@lists.xensource.com Subject Re: [XenPPC] systemsim-gpul problems On Thu, 2007-01-04 at 10:32 -0700, Yoder Stuart-B08248 wrote: I've been trying to get systemsim-gpul working with the latest Xen-PPC. The last changeset that worked was 7ad4645e7a54 (11/22/06). Changeset ce8c1e26b2ae (Early boot memory avoidance improvemnts) broke the simulator with the 'Could not allocate RTAS tree' / HANG error. With changeset 878ce1f78ad3 (Fix systemsim-gpul failure to boot) and later changesets the RTAS allocation / HANG problem is fixed but the simulator still won't boot into Linux. If I compare logs of the working and non-working Xen/Linux runs in the simulator, with the current Xen Linux hangs near the end of its boot. Last messages from Linux are: i2c /dev entries driver IPv4 over IPv4 tunneling driver TCP bic registered NET: Registered protocol family 1 NET: Registered protocol family 17 ...then nothing. Shortly after this point in the boot is where the RAMDISK is decompressed and accessed. I'm wondering if the boot related memory improvements have affected a RAMDISK built into Linux and Xen. Have you tried attaching GDB to systemsim to figure out what's going on? Has anyone else had recent changesets working on the simulator? I haven't tried simulator in quite some time... -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
RE: [XenPPC] systemsim-gpul problems
On Thu, 2007-01-04 at 16:48 -0700, Yoder Stuart-B08248 wrote: Whad'ya know...if you wait long enough it works! So simulator performance is acceptable until mid-way through dom0 boot? It would be good to figure out the source of the slowdown. -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
RE: [XenPPC] systemsim-gpul problems
Could be that midway through a Linux boot all code is sequential, but at a later point Linux tries to initialize its timing facilities. This involves linux spinning on the timebase register which forces systemsim to simulate many, many useless cycles. Beyond that Linux initialization sees a lot of cache initialization which requires memory zeroing, which also is not simulator friendly. I see this sort of behavior simply by running linux on the simulator wit or without a hypervisor. On Thu, 2007-01-04 at 18:19 -0600, Hollis Blanchard wrote: On Thu, 2007-01-04 at 16:48 -0700, Yoder Stuart-B08248 wrote: Whad'ya know...if you wait long enough it works! So simulator performance is acceptable until mid-way through dom0 boot? It would be good to figure out the source of the slowdown. -- Michal Ostrowski [EMAIL PROTECTED] ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
RE: [XenPPC] systemsim-gpul problems: reserved memory areas
On Thu, 2007-01-04 at 14:15 -0700, Yoder Stuart-B08248 wrote: I have turned on debug prints in arch/powerpc/boot_of.c. One thing I'm puzzling over is why boot_of_alloc_init() is marking overlapping regions of memory. Does that make sense?? It doesn't exactly make sense, but in this case it's not a real problem... [snip] boot_of_alloc_init: marking 0x0 - 0x0^M boot_of_alloc_init: marking 0x0 - 0x400^M These two come from your Open Firmware device tree. They don't make sense IMHO. systemsim may have a bug in its device tree. We could also have a bug in the way we parse the available property, but I believe our code has been tested on PowerMac and SLOF, and it looks correct to me. boot_of_alloc_init: marking 0x40 - 0x88b000^M This is our code reserving Xen's text, _start to _end. boot_of_alloc_init: marking 0x0 - 0x3000^M This is our code reserving the exception handlers. -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
