I haven't been following this thread very closely at all, but one concern when you do this is that the interrupt stuff is a bit vector and with 128 processors, your bit vector will no longer fit into a 64-bit word. This error could simply be that you're overflowing the 64-bit word into the next 64-bit word and the programmed IO isn't set up to respond to that word.
Nate On Wed, Aug 24, 2011 at 3:38 PM, Dibakar Gope <[email protected]> wrote: > Hi Ali, > > Thank you for your response. I made some changes as per your suggestion in > platform.S and src/dev/alpha/(backdoor|tsunami_cchip).*. As I mentioned in my > last post that I compiled the vmlinux with NR_CPU set to 128 and BIG_TSUNAMI > set to yes. In platform.S, the changes I made are: > > #if defined(BIG_TSUNAMI) > #define MAXPROC 0x7f > #define IPIQ_addr 0x800 > #define IPIQ_shift 0 > #define IPIR_addr 0x880 > #define IPIR_shift 0 > #define RTC_addr 0x8c0 > #define RTC_shift 0 > #define DIR_addr 0xa2 > > The changes made in access.h,backdoor.cc and tsunami_cchip.cc are: > access.h: 72: uint64_t cpuStack[128]; // 70: > backdoor.cc:182: if (cpunum >= 0 && cpunum < 128) > backdoor.cc:240: if (cpunum >= 0 && cpunum < 128) > backdoor.cc:270: SERIALIZE_ARRAY(cpuStack,128); > backdoor.cc:292: UNSERIALIZE_ARRAY(cpuStack, 128); > tsunami_cchip.cc:86: Addr regnum = (pkt->getAddr() - pioAddr) >> 7; > tsunami_cchip.cc:97: pkt->set(dim[(daddr >> 4) & 0x7F]); > tsunami_cchip.cc:103: pkt->set(dir[(daddr >> 4) & 0x7F]); > tsunami_cchip.cc:201: Addr regnum = (pkt->getAddr() - pioAddr) >> 7; > tsunami_cchip.cc:213: int number = (daddr >> 4) & 0x7F; > tsunami_cchip.cc:316: for(int x = 0; x < 128; x++) > > And in tsunami.hh, the Max_CPUs is set to 128. However with the above > changes, the simulation is hitting an unimplemented regnum case > (TSDEV_CC_MPR3, line no 362) in TsunamiCChip::write(PacketPtr pkt) function > in tsunami_cchip.cc. The regnum case is as follows: > case TSDEV_CC_MPR3: > panic("TSDEV_CC_MPRx write not implemented\n"); > > So in order to bypass this panic condition, I did add a break statement in > the above case so as not to hit the panic. But as a result, the > m5term/simulation output is stuck to the following point from the last one > day: > ...................................................................................... > ...................................................................................... > Bootstraping CPU 95 with sp=0xFFFFFC0000132000 > Bootstraping CPU 96 with sp=0xFFFFFC0000134000 > Bootstraping CPU 97 with sp=0xFFFFFC0000136000 > Bootstraping CPU 98 with sp=0xFFFFFC0000138000 > Bootstraping CPU 99 with sp=0xFFFFFC000013A000 > Bootstraping CPU 100 with sp=0xFFFFFC000013C000 > Bootstraping CPU 101 with sp=0xFFFFFC000013E000 > unix_boot_mem ends at FFFFFC0000140000 > k_argc = 0 > jumping to kernel at 0xFFFFFC0000310000, (PCBB 0xFFFFFC0000018180 pfn 1105) > CallbackFixup 0 18000, t7=FFFFFC0000814000 > Entering slaveloop for cpu 88 my_rpb=FFFFFC0000025D80 > EnEtenrteirningg sslalvaevleloooopp f ofro r ccpup u 9510 0my_rp > bmy=_rpb=FFFFFFFCF00FF0C00000206F00207B8 > > 0 > EnteErnitnering gsl avslealvoeloopo pfo rf ocr pcupu 9389 m > y_mrpy_brp=b=FFFFFFFFCF0F0C00000002600206A000 > > Entering slaveloop for cpu 90 my_rpb=FFFFFC0000026280 > EEntnetreirningg sslalvaevleloooop pf ofror ccpup u 9994 m > ym_ryp_brpb==FFFFFFCF00F0F0FC0020709000026 > 80 > C > EEntnteerriinEnngtg erisn lslagavv eeslllooaovpeo pl fofooorpr cfpocupru c > pu 929110 m1y_r pmb ym_yr=_prbpb==FFFFFFFCF0FF0FF0FCF00F00C0026007002806005 > 00 > 207E0 > > EnterEinnterEgin ntselrgaivn egsll oasovlpea lvfeoolorop o cpfp ourf o rc > pcupu 97969 m8y_r pmb > ym_yr=_prbpb==FFFFFFFFFFCFF0FF0FC0C0000000002700402027071688 > 0 > > > Do you think that I need to write the regnum cases for the cchip/big_tsunami > registers in TsunamiCChip::write(PacketPtr pkt) and > TsunamiCChip::read(PacketPtr pkt) function in regunm variable range of 40-7f, > just like the existing cases (00-3f). What do you think, what might be the > other plausible reasons for this so long halt in execution/simulation? > > Thanks! > Dibakar > PhD Student, UW-Madison > > > On 08/22/11, Ali Saidi wrote: >> Hi Dibakar, >> >> You'll need to just make something up that works and provides enough bits in >> each interrupt register to support 128 cores. There is no magic formula. >> You'll also need to make the complimentary changes to the kernel, palcode, >> and src/dev/alpha/(backdoor|tsunami_cchip).*. >> >> Ali >> >> >> >> On Aug 22, 2011, at 9:11 AM, Dibakar Gope wrote: >> >> > Hi, >> > I am trying to simulate more than 64 processors in M5 (ALPHA FS mode). As >> > of now, I could run the M5 simulations with upto 64 procs with necessary >> > changes in vmlinux kernel and updated BIG_TSUNAMI supported tsb_osfpal. In >> > order to obtain the updated tsb_osfpal, I used the changeset in Makefile >> > and platform.S (system/alpha/palcode, changeset given in >> > http://www.mail-archive.com/[email protected]/msg10402.html) and >> > cross-compiled them. However now I need to run simulations on more than 64 >> > procs. I cross-compiled the vmlinux kernel (2.6.27) with NR_CPUs set to >> > 128 and BIG_TSUNAMI set to yes. However, Soon after starting the >> > simulation, a continuous stream of "warn: clear IPI for CPU=#, but NO IPI" >> > warning messages are printed to the screen. From the prior post on that, >> > it seems an issue with the tsb_osfpal pal code (as it supports upto 64 >> > procs). So I was trying to change the config in Makefile and platform.S >> > (system/alpha/palcode) to make it compatible for 128, but in platfo >> rm >> > .S file, I could not figure out all the chnages I should make, for example >> > in system/alpha/palcode/platform_m5.S, >> > (changeset given in >> > http://www.mail-archive.com/[email protected]/msg10402.html) >> > @@ -71,6 +71,26 @@ >> > #define osfpcb_q_Ksp pcb_q_ksp >> > #define pal_impure_common_size ((0x200 + 7) & 0xfff8) >> > >> > +#ifdef BIG_TSUNAMI >> > +#define MAXPROC 0x3f -------------> changed to 0x7f >> > +#define IPIQ_addr 0x800 -------------> what will be the new >> > value for 128 procs??? >> > +#define IPIQ_shift 0 >> > +#define IPIR_addr 0x840 -------------> new value for 128 procs??? >> > +#define IPIR_shift 0 >> > +#define RTC_addr 0x880 -------------> new value for 128 procs??? >> > +#define RTC_shift 0 >> > +#define DIR_addr 0xa2 -------------> new value for 128 >> > procs??? >> > What should be the new values for those IPIQ_addr, IPIR_addr and DIR_addr >> > in order to support 128 procs? Are their any additional chages I should >> > make in the remaining code of the platform_m5.S? Could you please provide >> > me with some information/detail in making those changes in platform.S? Ali >> > might give me some pointers, as he made the prior changes in palcode. >> > Thanks! >> > Dibakar Gope >> > Graduate Student, UW-Madison >> > _______________________________________________ >> > gem5-users mailing list >> > [email protected] >> > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >> > >> >> _______________________________________________ >> gem5-users mailing list >> [email protected] >> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users > _______________________________________________ > gem5-users mailing list > [email protected] > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users > _______________________________________________ gem5-users mailing list [email protected] http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
