Re: [Qemu-devel] Poking a sun4v machine
Am 06.05.2012 12:58, schrieb Blue Swirl: > On Sun, May 6, 2012 at 10:18 AM, Andreas Färber wrote: >> Am 06.05.2012 10:58, schrieb Alexander Graf: >>> Am 06.05.2012 um 10:29 schrieb Blue Swirl : On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko wrote: > On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: >> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko >> wrote: >>> To me it looks like at the end do_interrupt will have less >>> common parts between sun4u and sun4v than specific ones. >> >> Same as tlb_fill(), switch() or function pointer. The functions are >> different. >> >> This is unavoidable (unless maybe in the future the TLB handling can >> be pushed partially higher so mmu_idx parameters can be eliminated) >> and the performance cost is not great. >> [...] Architectures are not meant to handle small issues like this. Should performance become a problem, there are a plenty of lower hanging fruits where to start optimizing. Even in this case, rather than the new architecture solution, it could be possible to build separate TLB handlers which call directly the correct MMU functions without switches and these would be selected at translation time or earlier. For the PPCEMB case, maybe the memory API could be changed to handle different page sizes without loss of performance, I don't know. Devices should not depend on TARGET_PAGE_SIZE. >>> >>> It's not a matter of an API. The main problem is that the QEMU TLB has to >>> be fine grained enough to handle 1k faults, so it has to be in 1k-steps in >>> its current design. >> >> Due to the TLB being a common property of CPUs yet dependent on target >> sizes, I had the idea of breaking it out from CPU(Arch)State so that we >> can have a different inheritance hierarchy than for CPUs. But that's >> just an unevaluated idea so far and more than 138 commits in the future. > > Yes, this is in part what I wanted with cputlb.[ch] change. > >> Same problem for breakpoints and watchpoints btw. >> >> ppcemb is no priority for me, but in order to move the very basic halted >> and interrupt_request fields to CPUState, for a VMState post_load hook >> we need to be able to tlb_flush() based on CPUState rather than >> CPUArchState. Currently just a pointer hack on qom-cpu branch; as an >> interim solution I might just defer that to target code to workaround >> the problem... > > Could the TLB become a separate object? That's what I meant with "breaking it out from CPU(Arch)State". :) /-F -- SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] Poking a sun4v machine
On Sun, May 6, 2012 at 10:18 AM, Andreas Färber wrote: > Am 06.05.2012 10:58, schrieb Alexander Graf: >> Am 06.05.2012 um 10:29 schrieb Blue Swirl : >>> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko >>> wrote: On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: > On Tue, May 1, 2012 at 13:54, Artyom Tarasenko > wrote: >> To me it looks like at the end do_interrupt will have less >> common parts between sun4u and sun4v than specific ones. > > Same as tlb_fill(), switch() or function pointer. The functions are > different. > > This is unavoidable (unless maybe in the future the TLB handling can > be pushed partially higher so mmu_idx parameters can be eliminated) > and the performance cost is not great. > [...] >>> Architectures are not meant to handle small issues like this. >>> Should performance become a problem, there are a plenty of lower >>> hanging fruits where to start optimizing. >>> >>> Even in this case, rather than the new architecture solution, it could >>> be possible to build separate TLB handlers which call directly the >>> correct MMU functions without switches and these would be selected at >>> translation time or earlier. For the PPCEMB case, maybe the memory API >>> could be changed to handle different page sizes without loss of >>> performance, I don't know. Devices should not depend on >>> TARGET_PAGE_SIZE. >> >> It's not a matter of an API. The main problem is that the QEMU TLB has to be >> fine grained enough to handle 1k faults, so it has to be in 1k-steps in its >> current design. > > Due to the TLB being a common property of CPUs yet dependent on target > sizes, I had the idea of breaking it out from CPU(Arch)State so that we > can have a different inheritance hierarchy than for CPUs. But that's > just an unevaluated idea so far and more than 138 commits in the future. Yes, this is in part what I wanted with cputlb.[ch] change. > Same problem for breakpoints and watchpoints btw. > > ppcemb is no priority for me, but in order to move the very basic halted > and interrupt_request fields to CPUState, for a VMState post_load hook > we need to be able to tlb_flush() based on CPUState rather than > CPUArchState. Currently just a pointer hack on qom-cpu branch; as an > interim solution I might just defer that to target code to workaround > the problem... Could the TLB become a separate object? The TLB functions could use a pointer to struct CPUCommonTLB (containing approximately what CPU_COMMON_TLB in cpu-defs.h has now) instead of CPUState. Though we need to pass a CPUState to TLB fill function anyway. > > Andreas > > -- > SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany > GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] Poking a sun4v machine
Am 06.05.2012 10:58, schrieb Alexander Graf: > Am 06.05.2012 um 10:29 schrieb Blue Swirl : >> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko wrote: >>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: On Tue, May 1, 2012 at 13:54, Artyom Tarasenko wrote: > To me it looks like at the end do_interrupt will have less > common parts between sun4u and sun4v than specific ones. Same as tlb_fill(), switch() or function pointer. The functions are different. This is unavoidable (unless maybe in the future the TLB handling can be pushed partially higher so mmu_idx parameters can be eliminated) and the performance cost is not great. [...] >> Architectures are not meant to handle small issues like this. >> Should performance become a problem, there are a plenty of lower >> hanging fruits where to start optimizing. >> >> Even in this case, rather than the new architecture solution, it could >> be possible to build separate TLB handlers which call directly the >> correct MMU functions without switches and these would be selected at >> translation time or earlier. For the PPCEMB case, maybe the memory API >> could be changed to handle different page sizes without loss of >> performance, I don't know. Devices should not depend on >> TARGET_PAGE_SIZE. > > It's not a matter of an API. The main problem is that the QEMU TLB has to be > fine grained enough to handle 1k faults, so it has to be in 1k-steps in its > current design. Due to the TLB being a common property of CPUs yet dependent on target sizes, I had the idea of breaking it out from CPU(Arch)State so that we can have a different inheritance hierarchy than for CPUs. But that's just an unevaluated idea so far and more than 138 commits in the future. Same problem for breakpoints and watchpoints btw. ppcemb is no priority for me, but in order to move the very basic halted and interrupt_request fields to CPUState, for a VMState post_load hook we need to be able to tlb_flush() based on CPUState rather than CPUArchState. Currently just a pointer hack on qom-cpu branch; as an interim solution I might just defer that to target code to workaround the problem... Andreas -- SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] Poking a sun4v machine
Am 06.05.2012 um 11:13 schrieb Blue Swirl : > On Sun, May 6, 2012 at 8:58 AM, Alexander Graf wrote: >> >> >> Am 06.05.2012 um 10:29 schrieb Blue Swirl : >> >>> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko >>> wrote: On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: > On Tue, May 1, 2012 at 13:54, Artyom Tarasenko > wrote: >> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl wrote: >>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko >>> wrote: On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: > Am 30.04.2012 18:39, schrieb Artyom Tarasenko: >> Tried to boot QEMU Niagara machine with the firmware from the >> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html >> ) >> , and it dies very early. >> The reason: in translate.c >> >> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >> >> and the dc->mem_idx is initialized like this: >> >> if (env1->tl > 0) { >> return MMU_NUCLEUS_IDX; >> } else if (cpu_hypervisor_mode(env1)) { >> return MMU_HYPV_IDX; >> } else if (cpu_supervisor_mode(env1)) { >> return MMU_KERNEL_IDX; >> } else { >> return MMU_USER_IDX; >> } >> >> Which seems to be conceptually incorrect. After reset tl == MAXTL, >> but >> still super- and hyper-visor bits are set, so both supervisor(dc) and >> hypervisor(dc) must return 1 which is impossible in the current >> implementation. >> >> What would be the proper way to fix it? Make mem_idx bitmap, add two >> more variables to DisasContext, or ...? >> >> Some other findings/questions: >> >> /* Sun4v generic Niagara machine */ >> { >> .default_cpu_model = "Sun UltraSparc T1", >> .console_serial_base = 0xfff0c2c000ULL, >> >> Where is this address coming from? The OpenSPARC Niagara machine has >> a >> "dumb serial" at 0x1f1000ULL. >> >> And the biggest issue: UA2005 (as well as UA2007) describe a totally >> different format for a MMU TTE entry than the one sun4u CPU are >> using. >> I think the best way to handle it would be splitting off Niagara >> machine, and #defining MMU bits differently for sun4u and sun4v >> machines. >> >> Do we the cases in qemu where more than two (qemu-system-xxx and >> qemu-system-xxx64) binaries are produced? >> Would the name qemu-system-sun4v fit the naming convention? > > We have such a case for ppc (ppcemb) and it is kind of a maintenance > nightmare - I'm working towards getting rid of it with my QOM CPU > work. > Better avoid it for sparc in the first place. > > Instead, you should add a callback function pointer to SPARCCPUClass > that you initialize based on CPU model so that is behaves differently > at > runtime rather than at compile time. > Or if it's just about the class_init then after the Hard Freeze I can > start polishing my subclasses for sparc so that you can add a special > class_init for Niagara. But this would mean that the defines from #define TTE_NFO_BIT (1ULL << 60) to #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) inclusive would need to be replaced with functions and variables? Sounds like a further performance regression for sun4u? >>> >>> There could be parallel definitions for sun4u (actually UltraSparc-III >>> onwards the MMU is again different) and sun4v. >>> >>> At tlb_fill(), different implementations can be selected based on MMU >>> model. For ASI accesses, we can add conditional code but for higher >>> performance, some checks can be moved to translation time. >> >> Can be done, but what is the gain of having it runtime configurable? > > I was thinking of code like this in: > > switch (env->mmu_model) { > case MMU_US2: > return tlb_fill_us2(..); > case MMU_US3: > return tlb_fill_us3(..); > case MMU_US4: > return tlb_fill_us4(..); > case MMU_T1: > return tlb_fill_t1(..); > case MMU_T2: > return tlb_fill_t2(..); > } > > The perfomance cost shouldn't be too high. Alternatively a function > pointer could be set up. Actually I was more worried about get_physical_address_* than filling, there we would have to use variables instead of constants and functions instead of macros. >>> >>> Preferably entirely different functions with consta
Re: [Qemu-devel] Poking a sun4v machine
On Sun, May 6, 2012 at 8:58 AM, Alexander Graf wrote: > > > Am 06.05.2012 um 10:29 schrieb Blue Swirl : > >> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko wrote: >>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: On Tue, May 1, 2012 at 13:54, Artyom Tarasenko wrote: > On Tue, May 1, 2012 at 11:25 AM, Blue Swirl wrote: >> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko >> wrote: >>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber >>> wrote: Am 30.04.2012 18:39, schrieb Artyom Tarasenko: > Tried to boot QEMU Niagara machine with the firmware from the > OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) > , and it dies very early. > The reason: in translate.c > > #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) > #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) > > and the dc->mem_idx is initialized like this: > > if (env1->tl > 0) { > return MMU_NUCLEUS_IDX; > } else if (cpu_hypervisor_mode(env1)) { > return MMU_HYPV_IDX; > } else if (cpu_supervisor_mode(env1)) { > return MMU_KERNEL_IDX; > } else { > return MMU_USER_IDX; > } > > Which seems to be conceptually incorrect. After reset tl == MAXTL, but > still super- and hyper-visor bits are set, so both supervisor(dc) and > hypervisor(dc) must return 1 which is impossible in the current > implementation. > > What would be the proper way to fix it? Make mem_idx bitmap, add two > more variables to DisasContext, or ...? > > Some other findings/questions: > > /* Sun4v generic Niagara machine */ > { > .default_cpu_model = "Sun UltraSparc T1", > .console_serial_base = 0xfff0c2c000ULL, > > Where is this address coming from? The OpenSPARC Niagara machine has a > "dumb serial" at 0x1f1000ULL. > > And the biggest issue: UA2005 (as well as UA2007) describe a totally > different format for a MMU TTE entry than the one sun4u CPU are using. > I think the best way to handle it would be splitting off Niagara > machine, and #defining MMU bits differently for sun4u and sun4v > machines. > > Do we the cases in qemu where more than two (qemu-system-xxx and > qemu-system-xxx64) binaries are produced? > Would the name qemu-system-sun4v fit the naming convention? We have such a case for ppc (ppcemb) and it is kind of a maintenance nightmare - I'm working towards getting rid of it with my QOM CPU work. Better avoid it for sparc in the first place. Instead, you should add a callback function pointer to SPARCCPUClass that you initialize based on CPU model so that is behaves differently at runtime rather than at compile time. Or if it's just about the class_init then after the Hard Freeze I can start polishing my subclasses for sparc so that you can add a special class_init for Niagara. >>> >>> But this would mean that the defines from >>> #define TTE_NFO_BIT (1ULL << 60) >>> to >>> #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) >>> >>> inclusive would need to be replaced with functions and variables? >>> Sounds like a further performance regression for sun4u? >> >> There could be parallel definitions for sun4u (actually UltraSparc-III >> onwards the MMU is again different) and sun4v. >> >> At tlb_fill(), different implementations can be selected based on MMU >> model. For ASI accesses, we can add conditional code but for higher >> performance, some checks can be moved to translation time. > > Can be done, but what is the gain of having it runtime configurable? I was thinking of code like this in: switch (env->mmu_model) { case MMU_US2: return tlb_fill_us2(..); case MMU_US3: return tlb_fill_us3(..); case MMU_US4: return tlb_fill_us4(..); case MMU_T1: return tlb_fill_t1(..); case MMU_T2: return tlb_fill_t2(..); } The perfomance cost shouldn't be too high. Alternatively a function pointer could be set up. >>> >>> Actually I was more worried about get_physical_address_* than filling, >>> there we would have to use variables instead of constants and >>> functions instead of macros. >> >> Preferably entirely different functions with constants. >> >>> Yes, we can always provide the register bank, older models just access some of those. > cpu_change_pstate should probably have another parameter (new_GL) > which is only valid for sun4v. > And, depending on a trap type,
Re: [Qemu-devel] Poking a sun4v machine
Am 06.05.2012 um 10:29 schrieb Blue Swirl : > On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko wrote: >> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: >>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko wrote: On Tue, May 1, 2012 at 11:25 AM, Blue Swirl wrote: > On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko > wrote: >> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: >>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko: Tried to boot QEMU Niagara machine with the firmware from the OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) , and it dies very early. The reason: in translate.c #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) and the dc->mem_idx is initialized like this: if (env1->tl > 0) { return MMU_NUCLEUS_IDX; } else if (cpu_hypervisor_mode(env1)) { return MMU_HYPV_IDX; } else if (cpu_supervisor_mode(env1)) { return MMU_KERNEL_IDX; } else { return MMU_USER_IDX; } Which seems to be conceptually incorrect. After reset tl == MAXTL, but still super- and hyper-visor bits are set, so both supervisor(dc) and hypervisor(dc) must return 1 which is impossible in the current implementation. What would be the proper way to fix it? Make mem_idx bitmap, add two more variables to DisasContext, or ...? Some other findings/questions: /* Sun4v generic Niagara machine */ { .default_cpu_model = "Sun UltraSparc T1", .console_serial_base = 0xfff0c2c000ULL, Where is this address coming from? The OpenSPARC Niagara machine has a "dumb serial" at 0x1f1000ULL. And the biggest issue: UA2005 (as well as UA2007) describe a totally different format for a MMU TTE entry than the one sun4u CPU are using. I think the best way to handle it would be splitting off Niagara machine, and #defining MMU bits differently for sun4u and sun4v machines. Do we the cases in qemu where more than two (qemu-system-xxx and qemu-system-xxx64) binaries are produced? Would the name qemu-system-sun4v fit the naming convention? >>> >>> We have such a case for ppc (ppcemb) and it is kind of a maintenance >>> nightmare - I'm working towards getting rid of it with my QOM CPU work. >>> Better avoid it for sparc in the first place. >>> >>> Instead, you should add a callback function pointer to SPARCCPUClass >>> that you initialize based on CPU model so that is behaves differently at >>> runtime rather than at compile time. >>> Or if it's just about the class_init then after the Hard Freeze I can >>> start polishing my subclasses for sparc so that you can add a special >>> class_init for Niagara. >> >> But this would mean that the defines from >> #define TTE_NFO_BIT (1ULL << 60) >> to >> #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) >> >> inclusive would need to be replaced with functions and variables? >> Sounds like a further performance regression for sun4u? > > There could be parallel definitions for sun4u (actually UltraSparc-III > onwards the MMU is again different) and sun4v. > > At tlb_fill(), different implementations can be selected based on MMU > model. For ASI accesses, we can add conditional code but for higher > performance, some checks can be moved to translation time. Can be done, but what is the gain of having it runtime configurable? >>> >>> I was thinking of code like this in: >>> >>> switch (env->mmu_model) { >>> case MMU_US2: >>> return tlb_fill_us2(..); >>> case MMU_US3: >>> return tlb_fill_us3(..); >>> case MMU_US4: >>> return tlb_fill_us4(..); >>> case MMU_T1: >>> return tlb_fill_t1(..); >>> case MMU_T2: >>> return tlb_fill_t2(..); >>> } >>> >>> The perfomance cost shouldn't be too high. Alternatively a function >>> pointer could be set up. >> >> Actually I was more worried about get_physical_address_* than filling, >> there we would have to use variables instead of constants and >> functions instead of macros. > > Preferably entirely different functions with constants. > >> >>> Yes, we can always provide the register bank, older models just access >>> some of those. >>> cpu_change_pstate should probably have another parameter (new_GL) which is only valid for sun4v. And, depending on a trap type, env->htba has to be taken instead of env->tbr. To me it looks like at the end do_interrupt will have less common parts between sun4u and sun4v than specific o
Re: [Qemu-devel] Poking a sun4v machine
On Wed, May 2, 2012 at 2:50 PM, Artyom Tarasenko wrote: > On Tue, May 1, 2012 at 3:54 PM, Blue Swirl wrote: >> On Tue, May 1, 2012 at 13:33, Artyom Tarasenko wrote: >>> On Tue, May 1, 2012 at 11:19 AM, Blue Swirl wrote: On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko wrote: > Tried to boot QEMU Niagara machine with the firmware from the > OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) > , and it dies very early. > The reason: in translate.c > > #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) > #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) > > and the dc->mem_idx is initialized like this: > > if (env1->tl > 0) { > return MMU_NUCLEUS_IDX; > } else if (cpu_hypervisor_mode(env1)) { > return MMU_HYPV_IDX; > } else if (cpu_supervisor_mode(env1)) { > return MMU_KERNEL_IDX; > } else { > return MMU_USER_IDX; > } > > Which seems to be conceptually incorrect. After reset tl == MAXTL, but > still super- and hyper-visor bits are set, so both supervisor(dc) and > hypervisor(dc) must return 1 which is impossible in the current > implementation. I don't think this is needed. The MMU index tells which TLB is used for guest virtual to host address translations, during tl == MAXTL we want to use hypervisor mode translations. >>> >>> Is just about the address translations? I thought it affects TB cache >>> as well, which has to be invalidated between nucleus, super- and >>> hypervisor switches? >> >> This is (should be) handled by cpu_get_tb_cpu_state(). But it looks >> like only supervisor mode (PS_PRIV) is taken into account, not others. >> >> Also when D/I MMU is enabled or disabled, TLB is flushed, so there's >> no need to store these bits. >> >> I'm not sure TL is interesting either. > > And what about the MMU contexts? UA2005 does have 3 ones: Primary, > Secondary and Nucleus. > Is it not relevant here? We also flush the TLB when the context registers are changed. This is not optimal, we'd want to add context numbers to TB flags but there's not enough space. New flags or context numbers should be added, they would be useful for other architectures too. Probably only primary context is interesting for Sparc guests, flushes for secondary and nucleus could be OK. Sparc32 MMU also has context numbers which is handled by flushing ATM. > > -- > Regards, > Artyom Tarasenko > > solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: [Qemu-devel] Poking a sun4v machine
On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko wrote: > On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: >> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko wrote: >>> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl wrote: On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko wrote: > On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: >> Am 30.04.2012 18:39, schrieb Artyom Tarasenko: >>> Tried to boot QEMU Niagara machine with the firmware from the >>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >>> , and it dies very early. >>> The reason: in translate.c >>> >>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >>> >>> and the dc->mem_idx is initialized like this: >>> >>> if (env1->tl > 0) { >>> return MMU_NUCLEUS_IDX; >>> } else if (cpu_hypervisor_mode(env1)) { >>> return MMU_HYPV_IDX; >>> } else if (cpu_supervisor_mode(env1)) { >>> return MMU_KERNEL_IDX; >>> } else { >>> return MMU_USER_IDX; >>> } >>> >>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >>> still super- and hyper-visor bits are set, so both supervisor(dc) and >>> hypervisor(dc) must return 1 which is impossible in the current >>> implementation. >>> >>> What would be the proper way to fix it? Make mem_idx bitmap, add two >>> more variables to DisasContext, or ...? >>> >>> Some other findings/questions: >>> >>> /* Sun4v generic Niagara machine */ >>> { >>> .default_cpu_model = "Sun UltraSparc T1", >>> .console_serial_base = 0xfff0c2c000ULL, >>> >>> Where is this address coming from? The OpenSPARC Niagara machine has a >>> "dumb serial" at 0x1f1000ULL. >>> >>> And the biggest issue: UA2005 (as well as UA2007) describe a totally >>> different format for a MMU TTE entry than the one sun4u CPU are using. >>> I think the best way to handle it would be splitting off Niagara >>> machine, and #defining MMU bits differently for sun4u and sun4v >>> machines. >>> >>> Do we the cases in qemu where more than two (qemu-system-xxx and >>> qemu-system-xxx64) binaries are produced? >>> Would the name qemu-system-sun4v fit the naming convention? >> >> We have such a case for ppc (ppcemb) and it is kind of a maintenance >> nightmare - I'm working towards getting rid of it with my QOM CPU work. >> Better avoid it for sparc in the first place. >> >> Instead, you should add a callback function pointer to SPARCCPUClass >> that you initialize based on CPU model so that is behaves differently at >> runtime rather than at compile time. >> Or if it's just about the class_init then after the Hard Freeze I can >> start polishing my subclasses for sparc so that you can add a special >> class_init for Niagara. > > But this would mean that the defines from > #define TTE_NFO_BIT (1ULL << 60) > to > #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) > > inclusive would need to be replaced with functions and variables? > Sounds like a further performance regression for sun4u? There could be parallel definitions for sun4u (actually UltraSparc-III onwards the MMU is again different) and sun4v. At tlb_fill(), different implementations can be selected based on MMU model. For ASI accesses, we can add conditional code but for higher performance, some checks can be moved to translation time. >>> >>> Can be done, but what is the gain of having it runtime configurable? >> >> I was thinking of code like this in: >> >> switch (env->mmu_model) { >> case MMU_US2: >> return tlb_fill_us2(..); >> case MMU_US3: >> return tlb_fill_us3(..); >> case MMU_US4: >> return tlb_fill_us4(..); >> case MMU_T1: >> return tlb_fill_t1(..); >> case MMU_T2: >> return tlb_fill_t2(..); >> } >> >> The perfomance cost shouldn't be too high. Alternatively a function >> pointer could be set up. > > Actually I was more worried about get_physical_address_* than filling, > there we would have to use variables instead of constants and > functions instead of macros. Preferably entirely different functions with constants. > >> Yes, we can always provide the register bank, older models just access >> some of those. >> >>> cpu_change_pstate should probably have another parameter (new_GL) >>> which is only valid for sun4v. >>> And, depending on a trap type, env->htba has to be taken instead of >>> env->tbr. To me it looks like at the end do_interrupt will have less >>> common parts between sun4u and sun4v than specific ones. >> >> Same as tlb_fill(), switch() or function pointer. The functions are >> different. >> >> This is unavoidable (unless maybe in the future the TLB handling can >> be pushed partially
Re: [Qemu-devel] Poking a sun4v machine
On Tue, May 1, 2012 at 3:54 PM, Blue Swirl wrote: > On Tue, May 1, 2012 at 13:33, Artyom Tarasenko wrote: >> On Tue, May 1, 2012 at 11:19 AM, Blue Swirl wrote: >>> On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko wrote: Tried to boot QEMU Niagara machine with the firmware from the OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) , and it dies very early. The reason: in translate.c #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) and the dc->mem_idx is initialized like this: if (env1->tl > 0) { return MMU_NUCLEUS_IDX; } else if (cpu_hypervisor_mode(env1)) { return MMU_HYPV_IDX; } else if (cpu_supervisor_mode(env1)) { return MMU_KERNEL_IDX; } else { return MMU_USER_IDX; } Which seems to be conceptually incorrect. After reset tl == MAXTL, but still super- and hyper-visor bits are set, so both supervisor(dc) and hypervisor(dc) must return 1 which is impossible in the current implementation. >>> >>> I don't think this is needed. The MMU index tells which TLB is used >>> for guest virtual to host address translations, during tl == MAXTL we >>> want to use hypervisor mode translations. >> >> Is just about the address translations? I thought it affects TB cache >> as well, which has to be invalidated between nucleus, super- and >> hypervisor switches? > > This is (should be) handled by cpu_get_tb_cpu_state(). But it looks > like only supervisor mode (PS_PRIV) is taken into account, not others. > > Also when D/I MMU is enabled or disabled, TLB is flushed, so there's > no need to store these bits. > > I'm not sure TL is interesting either. And what about the MMU contexts? UA2005 does have 3 ones: Primary, Secondary and Nucleus. Is it not relevant here? -- Regards, Artyom Tarasenko solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: [Qemu-devel] Poking a sun4v machine
On Tue, May 1, 2012 at 4:06 PM, Blue Swirl wrote: > On Tue, May 1, 2012 at 13:54, Artyom Tarasenko wrote: >> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl wrote: >>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko wrote: On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: > Am 30.04.2012 18:39, schrieb Artyom Tarasenko: >> Tried to boot QEMU Niagara machine with the firmware from the >> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >> , and it dies very early. >> The reason: in translate.c >> >> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >> >> and the dc->mem_idx is initialized like this: >> >> if (env1->tl > 0) { >> return MMU_NUCLEUS_IDX; >> } else if (cpu_hypervisor_mode(env1)) { >> return MMU_HYPV_IDX; >> } else if (cpu_supervisor_mode(env1)) { >> return MMU_KERNEL_IDX; >> } else { >> return MMU_USER_IDX; >> } >> >> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >> still super- and hyper-visor bits are set, so both supervisor(dc) and >> hypervisor(dc) must return 1 which is impossible in the current >> implementation. >> >> What would be the proper way to fix it? Make mem_idx bitmap, add two >> more variables to DisasContext, or ...? >> >> Some other findings/questions: >> >> /* Sun4v generic Niagara machine */ >> { >> .default_cpu_model = "Sun UltraSparc T1", >> .console_serial_base = 0xfff0c2c000ULL, >> >> Where is this address coming from? The OpenSPARC Niagara machine has a >> "dumb serial" at 0x1f1000ULL. >> >> And the biggest issue: UA2005 (as well as UA2007) describe a totally >> different format for a MMU TTE entry than the one sun4u CPU are using. >> I think the best way to handle it would be splitting off Niagara >> machine, and #defining MMU bits differently for sun4u and sun4v >> machines. >> >> Do we the cases in qemu where more than two (qemu-system-xxx and >> qemu-system-xxx64) binaries are produced? >> Would the name qemu-system-sun4v fit the naming convention? > > We have such a case for ppc (ppcemb) and it is kind of a maintenance > nightmare - I'm working towards getting rid of it with my QOM CPU work. > Better avoid it for sparc in the first place. > > Instead, you should add a callback function pointer to SPARCCPUClass > that you initialize based on CPU model so that is behaves differently at > runtime rather than at compile time. > Or if it's just about the class_init then after the Hard Freeze I can > start polishing my subclasses for sparc so that you can add a special > class_init for Niagara. But this would mean that the defines from #define TTE_NFO_BIT (1ULL << 60) to #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) inclusive would need to be replaced with functions and variables? Sounds like a further performance regression for sun4u? >>> >>> There could be parallel definitions for sun4u (actually UltraSparc-III >>> onwards the MMU is again different) and sun4v. >>> >>> At tlb_fill(), different implementations can be selected based on MMU >>> model. For ASI accesses, we can add conditional code but for higher >>> performance, some checks can be moved to translation time. >> >> Can be done, but what is the gain of having it runtime configurable? > > I was thinking of code like this in: > > switch (env->mmu_model) { > case MMU_US2: > return tlb_fill_us2(..); > case MMU_US3: > return tlb_fill_us3(..); > case MMU_US4: > return tlb_fill_us4(..); > case MMU_T1: > return tlb_fill_t1(..); > case MMU_T2: > return tlb_fill_t2(..); > } > > The perfomance cost shouldn't be too high. Alternatively a function > pointer could be set up. Actually I was more worried about get_physical_address_* than filling, there we would have to use variables instead of constants and functions instead of macros. > Yes, we can always provide the register bank, older models just access > some of those. > >> cpu_change_pstate should probably have another parameter (new_GL) >> which is only valid for sun4v. >> And, depending on a trap type, env->htba has to be taken instead of >> env->tbr. To me it looks like at the end do_interrupt will have less >> common parts between sun4u and sun4v than specific ones. > > Same as tlb_fill(), switch() or function pointer. The functions are different. > > This is unavoidable (unless maybe in the future the TLB handling can > be pushed partially higher so mmu_idx parameters can be eliminated) > and the performance cost is not great. So, altogether you'd still prefer run-time checks over having qemu-system-sun4v (or -sparc64v) ? -- Regards, Artyom Tarasenko solaris/sparc under qemu b
Re: [Qemu-devel] Poking a sun4v machine
On 30.04.2012, at 19:15, Andreas Färber wrote: > Am 30.04.2012 18:39, schrieb Artyom Tarasenko: >> Tried to boot QEMU Niagara machine with the firmware from the >> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >> , and it dies very early. >> The reason: in translate.c >> >> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >> >> and the dc->mem_idx is initialized like this: >> >>if (env1->tl > 0) { >>return MMU_NUCLEUS_IDX; >>} else if (cpu_hypervisor_mode(env1)) { >>return MMU_HYPV_IDX; >>} else if (cpu_supervisor_mode(env1)) { >>return MMU_KERNEL_IDX; >>} else { >>return MMU_USER_IDX; >>} >> >> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >> still super- and hyper-visor bits are set, so both supervisor(dc) and >> hypervisor(dc) must return 1 which is impossible in the current >> implementation. >> >> What would be the proper way to fix it? Make mem_idx bitmap, add two >> more variables to DisasContext, or ...? >> >> Some other findings/questions: >> >>/* Sun4v generic Niagara machine */ >>{ >>.default_cpu_model = "Sun UltraSparc T1", >>.console_serial_base = 0xfff0c2c000ULL, >> >> Where is this address coming from? The OpenSPARC Niagara machine has a >> "dumb serial" at 0x1f1000ULL. >> >> And the biggest issue: UA2005 (as well as UA2007) describe a totally >> different format for a MMU TTE entry than the one sun4u CPU are using. >> I think the best way to handle it would be splitting off Niagara >> machine, and #defining MMU bits differently for sun4u and sun4v >> machines. >> >> Do we the cases in qemu where more than two (qemu-system-xxx and >> qemu-system-xxx64) binaries are produced? >> Would the name qemu-system-sun4v fit the naming convention? > > We have such a case for ppc (ppcemb) and it is kind of a maintenance > nightmare - I'm working towards getting rid of it with my QOM CPU work. > Better avoid it for sparc in the first place. The big differentiator of ppcemb is that TARGET_PAGE_SIZE is 1k. That's a setting I really don't want to enable for anyone not running 440 code. Alex
Re: [Qemu-devel] Poking a sun4v machine
On Tue, May 1, 2012 at 13:54, Artyom Tarasenko wrote: > On Tue, May 1, 2012 at 11:25 AM, Blue Swirl wrote: >> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko wrote: >>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: Am 30.04.2012 18:39, schrieb Artyom Tarasenko: > Tried to boot QEMU Niagara machine with the firmware from the > OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) > , and it dies very early. > The reason: in translate.c > > #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) > #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) > > and the dc->mem_idx is initialized like this: > > if (env1->tl > 0) { > return MMU_NUCLEUS_IDX; > } else if (cpu_hypervisor_mode(env1)) { > return MMU_HYPV_IDX; > } else if (cpu_supervisor_mode(env1)) { > return MMU_KERNEL_IDX; > } else { > return MMU_USER_IDX; > } > > Which seems to be conceptually incorrect. After reset tl == MAXTL, but > still super- and hyper-visor bits are set, so both supervisor(dc) and > hypervisor(dc) must return 1 which is impossible in the current > implementation. > > What would be the proper way to fix it? Make mem_idx bitmap, add two > more variables to DisasContext, or ...? > > Some other findings/questions: > > /* Sun4v generic Niagara machine */ > { > .default_cpu_model = "Sun UltraSparc T1", > .console_serial_base = 0xfff0c2c000ULL, > > Where is this address coming from? The OpenSPARC Niagara machine has a > "dumb serial" at 0x1f1000ULL. > > And the biggest issue: UA2005 (as well as UA2007) describe a totally > different format for a MMU TTE entry than the one sun4u CPU are using. > I think the best way to handle it would be splitting off Niagara > machine, and #defining MMU bits differently for sun4u and sun4v > machines. > > Do we the cases in qemu where more than two (qemu-system-xxx and > qemu-system-xxx64) binaries are produced? > Would the name qemu-system-sun4v fit the naming convention? We have such a case for ppc (ppcemb) and it is kind of a maintenance nightmare - I'm working towards getting rid of it with my QOM CPU work. Better avoid it for sparc in the first place. Instead, you should add a callback function pointer to SPARCCPUClass that you initialize based on CPU model so that is behaves differently at runtime rather than at compile time. Or if it's just about the class_init then after the Hard Freeze I can start polishing my subclasses for sparc so that you can add a special class_init for Niagara. >>> >>> But this would mean that the defines from >>> #define TTE_NFO_BIT (1ULL << 60) >>> to >>> #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) >>> >>> inclusive would need to be replaced with functions and variables? >>> Sounds like a further performance regression for sun4u? >> >> There could be parallel definitions for sun4u (actually UltraSparc-III >> onwards the MMU is again different) and sun4v. >> >> At tlb_fill(), different implementations can be selected based on MMU >> model. For ASI accesses, we can add conditional code but for higher >> performance, some checks can be moved to translation time. > > Can be done, but what is the gain of having it runtime configurable? I was thinking of code like this in: switch (env->mmu_model) { case MMU_US2: return tlb_fill_us2(..); case MMU_US3: return tlb_fill_us3(..); case MMU_US4: return tlb_fill_us4(..); case MMU_T1: return tlb_fill_t1(..); case MMU_T2: return tlb_fill_t2(..); } The perfomance cost shouldn't be too high. Alternatively a function pointer could be set up. > >>> And would it be possible to have a different register set for an >>> inherited SPARCCPUClass ? >>> Also trap handling and related cpu registers behavior has to be quite >>> different. >> >> Not really, we already handle some (but not all cases). > > You mean by not handling AG? The UA2005 doesn't have IG and MG either, > but that's the easy part. Instead it has a few register banks > switchable with the GL register pointer. Yes, we can always provide the register bank, older models just access some of those. > cpu_change_pstate should probably have another parameter (new_GL) > which is only valid for sun4v. > And, depending on a trap type, env->htba has to be taken instead of > env->tbr. To me it looks like at the end do_interrupt will have less > common parts between sun4u and sun4v than specific ones. Same as tlb_fill(), switch() or function pointer. The functions are different. This is unavoidable (unless maybe in the future the TLB handling can be pushed partially higher so mmu_idx parameters can be eliminated) and the performance cost is not great. > Helpers such as cpu_hypervisor_mode() will need to be c
Re: [Qemu-devel] Poking a sun4v machine
On Tue, May 1, 2012 at 10:35 AM, Andreas Färber wrote: > Am 30.04.2012 19:38, schrieb Artyom Tarasenko: >> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: >>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko: Tried to boot QEMU Niagara machine with the firmware from the OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) , and it dies very early. The reason: in translate.c #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) and the dc->mem_idx is initialized like this: if (env1->tl > 0) { return MMU_NUCLEUS_IDX; } else if (cpu_hypervisor_mode(env1)) { return MMU_HYPV_IDX; } else if (cpu_supervisor_mode(env1)) { return MMU_KERNEL_IDX; } else { return MMU_USER_IDX; } Which seems to be conceptually incorrect. After reset tl == MAXTL, but still super- and hyper-visor bits are set, so both supervisor(dc) and hypervisor(dc) must return 1 which is impossible in the current implementation. What would be the proper way to fix it? Make mem_idx bitmap, add two more variables to DisasContext, or ...? Some other findings/questions: /* Sun4v generic Niagara machine */ { .default_cpu_model = "Sun UltraSparc T1", .console_serial_base = 0xfff0c2c000ULL, Where is this address coming from? The OpenSPARC Niagara machine has a "dumb serial" at 0x1f1000ULL. And the biggest issue: UA2005 (as well as UA2007) describe a totally different format for a MMU TTE entry than the one sun4u CPU are using. I think the best way to handle it would be splitting off Niagara machine, and #defining MMU bits differently for sun4u and sun4v machines. Do we the cases in qemu where more than two (qemu-system-xxx and qemu-system-xxx64) binaries are produced? Would the name qemu-system-sun4v fit the naming convention? >>> >>> We have such a case for ppc (ppcemb) and it is kind of a maintenance >>> nightmare - I'm working towards getting rid of it with my QOM CPU work. >>> Better avoid it for sparc in the first place. >>> >>> Instead, you should add a callback function pointer to SPARCCPUClass >>> that you initialize based on CPU model so that is behaves differently at >>> runtime rather than at compile time. >>> Or if it's just about the class_init then after the Hard Freeze I can >>> start polishing my subclasses for sparc so that you can add a special >>> class_init for Niagara. >> >> But this would mean that the defines from >> #define TTE_NFO_BIT (1ULL << 60) >> to >> #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) >> >> inclusive would need to be replaced with functions and variables? >> Sounds like a further performance regression for sun4u? > > First of all, I wouldn't use the term performance regression for a > target such as sparc64 that has never really worked before. ;-) Well, it works good enough for me. ;-) Also until the recent PCI changes it was able to boot HelenOS out of the box. > I don't know the details of the SPARC implementation or architecture. > What I am telling you is that there are ways to implement sun4u and > sun4v inside one executable; whether you use a new function or extend a > macro with an additional parameter is up to you and Blue. My guess is > that unlike ARM you won't need to mix different cores at runtime for the > time being. > > In ARM we have a macro IS_M() that distinguishes between the Cortex-M > and Cortex-A families rather than moving Cortex-M to its own executable. > The Freescale Vybrid MCUs are going to combine the two in one SoC, i.e. > in one qemu-system-arm executable simultaneously. > >> And would it be possible to have a different register set for an >> inherited SPARCCPUClass ? > > You can add new state fields to an inherited SPARCCPU. Can the inheritance hierarchy have any depth? Then we could think of having SPARC->(SPARC32,SPARC64) and SPARC64 -> (sun4u, sun4v, Fujitsu). But I still don't see the advantages of having it configurable at runtime. > If you need them in TCG as offsets from env, then CPUSPARCState might be > an easier place to add them but there you cannot distinguish between the > models. > > To an inherited SPARCCPUClass you can add fields for saving constant, > e.g., reset values for registers of the model. > > The unanswered question is whether SPARC should follow the x86 model > with declarative (and user-specified) definitions (in that case > extending SPARCCPUClass) or the ARM model with imperative per-model init > functions (in that case probably not extending SPARCCPU at all). > > Andreas > > -- > SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany > GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg -- Regards, Artyom Tarasenko solaris/sp
Re: [Qemu-devel] Poking a sun4v machine
On Tue, May 1, 2012 at 11:25 AM, Blue Swirl wrote: > On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko wrote: >> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: >>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko: Tried to boot QEMU Niagara machine with the firmware from the OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) , and it dies very early. The reason: in translate.c #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) and the dc->mem_idx is initialized like this: if (env1->tl > 0) { return MMU_NUCLEUS_IDX; } else if (cpu_hypervisor_mode(env1)) { return MMU_HYPV_IDX; } else if (cpu_supervisor_mode(env1)) { return MMU_KERNEL_IDX; } else { return MMU_USER_IDX; } Which seems to be conceptually incorrect. After reset tl == MAXTL, but still super- and hyper-visor bits are set, so both supervisor(dc) and hypervisor(dc) must return 1 which is impossible in the current implementation. What would be the proper way to fix it? Make mem_idx bitmap, add two more variables to DisasContext, or ...? Some other findings/questions: /* Sun4v generic Niagara machine */ { .default_cpu_model = "Sun UltraSparc T1", .console_serial_base = 0xfff0c2c000ULL, Where is this address coming from? The OpenSPARC Niagara machine has a "dumb serial" at 0x1f1000ULL. And the biggest issue: UA2005 (as well as UA2007) describe a totally different format for a MMU TTE entry than the one sun4u CPU are using. I think the best way to handle it would be splitting off Niagara machine, and #defining MMU bits differently for sun4u and sun4v machines. Do we the cases in qemu where more than two (qemu-system-xxx and qemu-system-xxx64) binaries are produced? Would the name qemu-system-sun4v fit the naming convention? >>> >>> We have such a case for ppc (ppcemb) and it is kind of a maintenance >>> nightmare - I'm working towards getting rid of it with my QOM CPU work. >>> Better avoid it for sparc in the first place. >>> >>> Instead, you should add a callback function pointer to SPARCCPUClass >>> that you initialize based on CPU model so that is behaves differently at >>> runtime rather than at compile time. >>> Or if it's just about the class_init then after the Hard Freeze I can >>> start polishing my subclasses for sparc so that you can add a special >>> class_init for Niagara. >> >> But this would mean that the defines from >> #define TTE_NFO_BIT (1ULL << 60) >> to >> #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) >> >> inclusive would need to be replaced with functions and variables? >> Sounds like a further performance regression for sun4u? > > There could be parallel definitions for sun4u (actually UltraSparc-III > onwards the MMU is again different) and sun4v. > > At tlb_fill(), different implementations can be selected based on MMU > model. For ASI accesses, we can add conditional code but for higher > performance, some checks can be moved to translation time. Can be done, but what is the gain of having it runtime configurable? >> And would it be possible to have a different register set for an >> inherited SPARCCPUClass ? >> Also trap handling and related cpu registers behavior has to be quite >> different. > > Not really, we already handle some (but not all cases). You mean by not handling AG? The UA2005 doesn't have IG and MG either, but that's the easy part. Instead it has a few register banks switchable with the GL register pointer. cpu_change_pstate should probably have another parameter (new_GL) which is only valid for sun4v. And, depending on a trap type, env->htba has to be taken instead of env->tbr. To me it looks like at the end do_interrupt will have less common parts between sun4u and sun4v than specific ones. >>> Helpers such as cpu_hypervisor_mode() will need to be changed to take a >>> SPARCCPU *cpu rather than CPUSPARCState *env argument; as an interim >>> solution sparc_env_get_cpu() can be used. (examples on the list for sh4) >>> >>> Andreas >>> >>> -- >>> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany >>> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg >> -- Regards, Artyom Tarasenko solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: [Qemu-devel] Poking a sun4v machine
On Tue, May 1, 2012 at 13:33, Artyom Tarasenko wrote: > On Tue, May 1, 2012 at 11:19 AM, Blue Swirl wrote: >> On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko wrote: >>> Tried to boot QEMU Niagara machine with the firmware from the >>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >>> , and it dies very early. >>> The reason: in translate.c >>> >>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >>> >>> and the dc->mem_idx is initialized like this: >>> >>> if (env1->tl > 0) { >>> return MMU_NUCLEUS_IDX; >>> } else if (cpu_hypervisor_mode(env1)) { >>> return MMU_HYPV_IDX; >>> } else if (cpu_supervisor_mode(env1)) { >>> return MMU_KERNEL_IDX; >>> } else { >>> return MMU_USER_IDX; >>> } >>> >>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >>> still super- and hyper-visor bits are set, so both supervisor(dc) and >>> hypervisor(dc) must return 1 which is impossible in the current >>> implementation. >> >> I don't think this is needed. The MMU index tells which TLB is used >> for guest virtual to host address translations, during tl == MAXTL we >> want to use hypervisor mode translations. > > Is just about the address translations? I thought it affects TB cache > as well, which has to be invalidated between nucleus, super- and > hypervisor switches? This is (should be) handled by cpu_get_tb_cpu_state(). But it looks like only supervisor mode (PS_PRIV) is taken into account, not others. Also when D/I MMU is enabled or disabled, TLB is flushed, so there's no need to store these bits. I'm not sure TL is interesting either. >>> What would be the proper way to fix it? Make mem_idx bitmap, add two >>> more variables to DisasContext, or ...? >>>
Re: [Qemu-devel] Poking a sun4v machine
On Tue, May 1, 2012 at 11:19 AM, Blue Swirl wrote: > On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko wrote: >> Tried to boot QEMU Niagara machine with the firmware from the >> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >> , and it dies very early. >> The reason: in translate.c >> >> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >> >> and the dc->mem_idx is initialized like this: >> >> if (env1->tl > 0) { >> return MMU_NUCLEUS_IDX; >> } else if (cpu_hypervisor_mode(env1)) { >> return MMU_HYPV_IDX; >> } else if (cpu_supervisor_mode(env1)) { >> return MMU_KERNEL_IDX; >> } else { >> return MMU_USER_IDX; >> } >> >> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >> still super- and hyper-visor bits are set, so both supervisor(dc) and >> hypervisor(dc) must return 1 which is impossible in the current >> implementation. > > I don't think this is needed. The MMU index tells which TLB is used > for guest virtual to host address translations, during tl == MAXTL we > want to use hypervisor mode translations. Is just about the address translations? I thought it affects TB cache as well, which has to be invalidated between nucleus, super- and hypervisor switches? >> What would be the proper way to fix it? Make mem_idx bitmap, add two >> more variables to DisasContext, or ...? >>
Re: [Qemu-devel] Poking a sun4v machine
On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko wrote: > On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: >> Am 30.04.2012 18:39, schrieb Artyom Tarasenko: >>> Tried to boot QEMU Niagara machine with the firmware from the >>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >>> , and it dies very early. >>> The reason: in translate.c >>> >>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >>> >>> and the dc->mem_idx is initialized like this: >>> >>> if (env1->tl > 0) { >>> return MMU_NUCLEUS_IDX; >>> } else if (cpu_hypervisor_mode(env1)) { >>> return MMU_HYPV_IDX; >>> } else if (cpu_supervisor_mode(env1)) { >>> return MMU_KERNEL_IDX; >>> } else { >>> return MMU_USER_IDX; >>> } >>> >>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >>> still super- and hyper-visor bits are set, so both supervisor(dc) and >>> hypervisor(dc) must return 1 which is impossible in the current >>> implementation. >>> >>> What would be the proper way to fix it? Make mem_idx bitmap, add two >>> more variables to DisasContext, or ...? >>> >>> Some other findings/questions: >>> >>> /* Sun4v generic Niagara machine */ >>> { >>> .default_cpu_model = "Sun UltraSparc T1", >>> .console_serial_base = 0xfff0c2c000ULL, >>> >>> Where is this address coming from? The OpenSPARC Niagara machine has a >>> "dumb serial" at 0x1f1000ULL. >>> >>> And the biggest issue: UA2005 (as well as UA2007) describe a totally >>> different format for a MMU TTE entry than the one sun4u CPU are using. >>> I think the best way to handle it would be splitting off Niagara >>> machine, and #defining MMU bits differently for sun4u and sun4v >>> machines. >>> >>> Do we the cases in qemu where more than two (qemu-system-xxx and >>> qemu-system-xxx64) binaries are produced? >>> Would the name qemu-system-sun4v fit the naming convention? >> >> We have such a case for ppc (ppcemb) and it is kind of a maintenance >> nightmare - I'm working towards getting rid of it with my QOM CPU work. >> Better avoid it for sparc in the first place. >> >> Instead, you should add a callback function pointer to SPARCCPUClass >> that you initialize based on CPU model so that is behaves differently at >> runtime rather than at compile time. >> Or if it's just about the class_init then after the Hard Freeze I can >> start polishing my subclasses for sparc so that you can add a special >> class_init for Niagara. > > But this would mean that the defines from > #define TTE_NFO_BIT (1ULL << 60) > to > #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) > > inclusive would need to be replaced with functions and variables? > Sounds like a further performance regression for sun4u? There could be parallel definitions for sun4u (actually UltraSparc-III onwards the MMU is again different) and sun4v. At tlb_fill(), different implementations can be selected based on MMU model. For ASI accesses, we can add conditional code but for higher performance, some checks can be moved to translation time. > > And would it be possible to have a different register set for an > inherited SPARCCPUClass ? > Also trap handling and related cpu registers behavior has to be quite > different. Not really, we already handle some (but not all cases). > > Artyom > >> Helpers such as cpu_hypervisor_mode() will need to be changed to take a >> SPARCCPU *cpu rather than CPUSPARCState *env argument; as an interim >> solution sparc_env_get_cpu() can be used. (examples on the list for sh4) >> >> Andreas >> >> -- >> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany >> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg > > > > -- > Regards, > Artyom Tarasenko > > solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: [Qemu-devel] Poking a sun4v machine
On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko wrote: > Tried to boot QEMU Niagara machine with the firmware from the > OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) > , and it dies very early. > The reason: in translate.c > > #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) > #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) > > and the dc->mem_idx is initialized like this: > > if (env1->tl > 0) { > return MMU_NUCLEUS_IDX; > } else if (cpu_hypervisor_mode(env1)) { > return MMU_HYPV_IDX; > } else if (cpu_supervisor_mode(env1)) { > return MMU_KERNEL_IDX; > } else { > return MMU_USER_IDX; > } > > Which seems to be conceptually incorrect. After reset tl == MAXTL, but > still super- and hyper-visor bits are set, so both supervisor(dc) and > hypervisor(dc) must return 1 which is impossible in the current > implementation. I don't think this is needed. The MMU index tells which TLB is used for guest virtual to host address translations, during tl == MAXTL we want to use hypervisor mode translations. > > What would be the proper way to fix it? Make mem_idx bitmap, add two > more variables to DisasContext, or ...? > > Some other findings/questions: > > /* Sun4v generic Niagara machine */ > { > .default_cpu_model = "Sun UltraSparc T1", > .console_serial_base = 0xfff0c2c000ULL, > > Where is this address coming from? The OpenSPARC Niagara machine has a > "dumb serial" at 0x1f1000ULL. I think I actually used Ontario machine definitions. > > And the biggest issue: UA2005 (as well as UA2007) describe a totally > different format for a MMU TTE entry than the one sun4u CPU are using. > I think the best way to handle it would be splitting off Niagara > machine, and #defining MMU bits differently for sun4u and sun4v > machines. > > Do we the cases in qemu where more than two (qemu-system-xxx and > qemu-system-xxx64) binaries are produced? > Would the name qemu-system-sun4v fit the naming convention? > > Artyom > > -- > Regards, > Artyom Tarasenko > > solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: [Qemu-devel] Poking a sun4v machine
Am 30.04.2012 19:38, schrieb Artyom Tarasenko: > On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: >> Am 30.04.2012 18:39, schrieb Artyom Tarasenko: >>> Tried to boot QEMU Niagara machine with the firmware from the >>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >>> , and it dies very early. >>> The reason: in translate.c >>> >>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >>> >>> and the dc->mem_idx is initialized like this: >>> >>> if (env1->tl > 0) { >>> return MMU_NUCLEUS_IDX; >>> } else if (cpu_hypervisor_mode(env1)) { >>> return MMU_HYPV_IDX; >>> } else if (cpu_supervisor_mode(env1)) { >>> return MMU_KERNEL_IDX; >>> } else { >>> return MMU_USER_IDX; >>> } >>> >>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >>> still super- and hyper-visor bits are set, so both supervisor(dc) and >>> hypervisor(dc) must return 1 which is impossible in the current >>> implementation. >>> >>> What would be the proper way to fix it? Make mem_idx bitmap, add two >>> more variables to DisasContext, or ...? >>> >>> Some other findings/questions: >>> >>> /* Sun4v generic Niagara machine */ >>> { >>> .default_cpu_model = "Sun UltraSparc T1", >>> .console_serial_base = 0xfff0c2c000ULL, >>> >>> Where is this address coming from? The OpenSPARC Niagara machine has a >>> "dumb serial" at 0x1f1000ULL. >>> >>> And the biggest issue: UA2005 (as well as UA2007) describe a totally >>> different format for a MMU TTE entry than the one sun4u CPU are using. >>> I think the best way to handle it would be splitting off Niagara >>> machine, and #defining MMU bits differently for sun4u and sun4v >>> machines. >>> >>> Do we the cases in qemu where more than two (qemu-system-xxx and >>> qemu-system-xxx64) binaries are produced? >>> Would the name qemu-system-sun4v fit the naming convention? >> >> We have such a case for ppc (ppcemb) and it is kind of a maintenance >> nightmare - I'm working towards getting rid of it with my QOM CPU work. >> Better avoid it for sparc in the first place. >> >> Instead, you should add a callback function pointer to SPARCCPUClass >> that you initialize based on CPU model so that is behaves differently at >> runtime rather than at compile time. >> Or if it's just about the class_init then after the Hard Freeze I can >> start polishing my subclasses for sparc so that you can add a special >> class_init for Niagara. > > But this would mean that the defines from > #define TTE_NFO_BIT (1ULL << 60) > to > #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) > > inclusive would need to be replaced with functions and variables? > Sounds like a further performance regression for sun4u? First of all, I wouldn't use the term performance regression for a target such as sparc64 that has never really worked before. ;-) I don't know the details of the SPARC implementation or architecture. What I am telling you is that there are ways to implement sun4u and sun4v inside one executable; whether you use a new function or extend a macro with an additional parameter is up to you and Blue. My guess is that unlike ARM you won't need to mix different cores at runtime for the time being. In ARM we have a macro IS_M() that distinguishes between the Cortex-M and Cortex-A families rather than moving Cortex-M to its own executable. The Freescale Vybrid MCUs are going to combine the two in one SoC, i.e. in one qemu-system-arm executable simultaneously. > And would it be possible to have a different register set for an > inherited SPARCCPUClass ? You can add new state fields to an inherited SPARCCPU. If you need them in TCG as offsets from env, then CPUSPARCState might be an easier place to add them but there you cannot distinguish between the models. To an inherited SPARCCPUClass you can add fields for saving constant, e.g., reset values for registers of the model. The unanswered question is whether SPARC should follow the x86 model with declarative (and user-specified) definitions (in that case extending SPARCCPUClass) or the ARM model with imperative per-model init functions (in that case probably not extending SPARCCPU at all). Andreas -- SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] Poking a sun4v machine
On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber wrote: > Am 30.04.2012 18:39, schrieb Artyom Tarasenko: >> Tried to boot QEMU Niagara machine with the firmware from the >> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) >> , and it dies very early. >> The reason: in translate.c >> >> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) >> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) >> >> and the dc->mem_idx is initialized like this: >> >> if (env1->tl > 0) { >> return MMU_NUCLEUS_IDX; >> } else if (cpu_hypervisor_mode(env1)) { >> return MMU_HYPV_IDX; >> } else if (cpu_supervisor_mode(env1)) { >> return MMU_KERNEL_IDX; >> } else { >> return MMU_USER_IDX; >> } >> >> Which seems to be conceptually incorrect. After reset tl == MAXTL, but >> still super- and hyper-visor bits are set, so both supervisor(dc) and >> hypervisor(dc) must return 1 which is impossible in the current >> implementation. >> >> What would be the proper way to fix it? Make mem_idx bitmap, add two >> more variables to DisasContext, or ...? >> >> Some other findings/questions: >> >> /* Sun4v generic Niagara machine */ >> { >> .default_cpu_model = "Sun UltraSparc T1", >> .console_serial_base = 0xfff0c2c000ULL, >> >> Where is this address coming from? The OpenSPARC Niagara machine has a >> "dumb serial" at 0x1f1000ULL. >> >> And the biggest issue: UA2005 (as well as UA2007) describe a totally >> different format for a MMU TTE entry than the one sun4u CPU are using. >> I think the best way to handle it would be splitting off Niagara >> machine, and #defining MMU bits differently for sun4u and sun4v >> machines. >> >> Do we the cases in qemu where more than two (qemu-system-xxx and >> qemu-system-xxx64) binaries are produced? >> Would the name qemu-system-sun4v fit the naming convention? > > We have such a case for ppc (ppcemb) and it is kind of a maintenance > nightmare - I'm working towards getting rid of it with my QOM CPU work. > Better avoid it for sparc in the first place. > > Instead, you should add a callback function pointer to SPARCCPUClass > that you initialize based on CPU model so that is behaves differently at > runtime rather than at compile time. > Or if it's just about the class_init then after the Hard Freeze I can > start polishing my subclasses for sparc so that you can add a special > class_init for Niagara. But this would mean that the defines from #define TTE_NFO_BIT (1ULL << 60) to #define TTE_PGSIZE(tte) (((tte) >> 61) & 3ULL) inclusive would need to be replaced with functions and variables? Sounds like a further performance regression for sun4u? And would it be possible to have a different register set for an inherited SPARCCPUClass ? Also trap handling and related cpu registers behavior has to be quite different. Artyom > Helpers such as cpu_hypervisor_mode() will need to be changed to take a > SPARCCPU *cpu rather than CPUSPARCState *env argument; as an interim > solution sparc_env_get_cpu() can be used. (examples on the list for sh4) > > Andreas > > -- > SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany > GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg -- Regards, Artyom Tarasenko solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: [Qemu-devel] Poking a sun4v machine
Am 30.04.2012 18:39, schrieb Artyom Tarasenko: > Tried to boot QEMU Niagara machine with the firmware from the > OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) > , and it dies very early. > The reason: in translate.c > > #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) > #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) > > and the dc->mem_idx is initialized like this: > > if (env1->tl > 0) { > return MMU_NUCLEUS_IDX; > } else if (cpu_hypervisor_mode(env1)) { > return MMU_HYPV_IDX; > } else if (cpu_supervisor_mode(env1)) { > return MMU_KERNEL_IDX; > } else { > return MMU_USER_IDX; > } > > Which seems to be conceptually incorrect. After reset tl == MAXTL, but > still super- and hyper-visor bits are set, so both supervisor(dc) and > hypervisor(dc) must return 1 which is impossible in the current > implementation. > > What would be the proper way to fix it? Make mem_idx bitmap, add two > more variables to DisasContext, or ...? > > Some other findings/questions: > > /* Sun4v generic Niagara machine */ > { > .default_cpu_model = "Sun UltraSparc T1", > .console_serial_base = 0xfff0c2c000ULL, > > Where is this address coming from? The OpenSPARC Niagara machine has a > "dumb serial" at 0x1f1000ULL. > > And the biggest issue: UA2005 (as well as UA2007) describe a totally > different format for a MMU TTE entry than the one sun4u CPU are using. > I think the best way to handle it would be splitting off Niagara > machine, and #defining MMU bits differently for sun4u and sun4v > machines. > > Do we the cases in qemu where more than two (qemu-system-xxx and > qemu-system-xxx64) binaries are produced? > Would the name qemu-system-sun4v fit the naming convention? We have such a case for ppc (ppcemb) and it is kind of a maintenance nightmare - I'm working towards getting rid of it with my QOM CPU work. Better avoid it for sparc in the first place. Instead, you should add a callback function pointer to SPARCCPUClass that you initialize based on CPU model so that is behaves differently at runtime rather than at compile time. Or if it's just about the class_init then after the Hard Freeze I can start polishing my subclasses for sparc so that you can add a special class_init for Niagara. Helpers such as cpu_hypervisor_mode() will need to be changed to take a SPARCCPU *cpu rather than CPUSPARCState *env argument; as an interim solution sparc_env_get_cpu() can be used. (examples on the list for sh4) Andreas -- SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
[Qemu-devel] Poking a sun4v machine
Tried to boot QEMU Niagara machine with the firmware from the OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html ) , and it dies very early. The reason: in translate.c #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX) #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX) and the dc->mem_idx is initialized like this: if (env1->tl > 0) { return MMU_NUCLEUS_IDX; } else if (cpu_hypervisor_mode(env1)) { return MMU_HYPV_IDX; } else if (cpu_supervisor_mode(env1)) { return MMU_KERNEL_IDX; } else { return MMU_USER_IDX; } Which seems to be conceptually incorrect. After reset tl == MAXTL, but still super- and hyper-visor bits are set, so both supervisor(dc) and hypervisor(dc) must return 1 which is impossible in the current implementation. What would be the proper way to fix it? Make mem_idx bitmap, add two more variables to DisasContext, or ...? Some other findings/questions: /* Sun4v generic Niagara machine */ { .default_cpu_model = "Sun UltraSparc T1", .console_serial_base = 0xfff0c2c000ULL, Where is this address coming from? The OpenSPARC Niagara machine has a "dumb serial" at 0x1f1000ULL. And the biggest issue: UA2005 (as well as UA2007) describe a totally different format for a MMU TTE entry than the one sun4u CPU are using. I think the best way to handle it would be splitting off Niagara machine, and #defining MMU bits differently for sun4u and sun4v machines. Do we the cases in qemu where more than two (qemu-system-xxx and qemu-system-xxx64) binaries are produced? Would the name qemu-system-sun4v fit the naming convention? Artyom -- Regards, Artyom Tarasenko solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu