I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms.
The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev <[email protected] > wrote: > That's not actually extending the TSS limit, that's what it works out to be > with the granularity bit set. The AM and WP bits were set to the wrong > thing according to the comments next to them I'm pretty sure. If we wanted > the other behavior, we might be able to change them back and have it work. > The _BASE registers hold the base of segments as they're specified by the > ISA. The _EFF_BASE registers hold the base that will actually be used in > address calculations based on the mode of the CPU. For instance, if you're > in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in > another mode. The _EFF_BASE would be 0 though, since the DS base is ignored > in that case. _EFF_BASE may not be used by the KVM CPU, but it should be > set up anyway in case we switch back to a regular CPU. > > Gabe > > On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev < > [email protected]> wrote: > > > Thank you for all the clarifications. I see that for SE to work on vmx > the > > TSS limit had to be extended, am and wp bits in CR0 had to be reset and > > *_EFF_BASE registers had be set in addition to *_BASE registers for TR > TSG > > IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR > > register (which gets passed to kvm) are the same if with or without > > *_EFF_BASE registers set. > > > > Thank you, > > Alex > > > > -----Original Message----- > > From: gem5-dev [mailto:[email protected]] On Behalf Of Gabe > Black > > via gem5-dev > > Sent: Wednesday, December 10, 2014 1:21 AM > > To: gem5 Developer List > > Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) > > > > Ok, I got SE working too. I'll clean up my patch and send that out in a > > bit. > > > > Gabe > > > > On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black <[email protected]> wrote: > > > > > I figured out what the other problem was, so here's the review. > > > > > > http://reviews.gem5.org/r/2557/ > > > > > > Gabe > > > > > > On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black <[email protected]> > wrote: > > > > > >> It was attached in my sent mail. Maybe it's being blocked by > something? > > >> I'm hunting down another problem so I don't want to move my tree > > >> around too much, but once that's done I'll post it as a review. > > >> > > >> Gabe > > >> > > >> On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev < > > >> [email protected]> wrote: > > >> > > >>> I haven't received any attachment to your email. So I don't have > > >>> your patch. > > >>> > > >>> Alex > > >>> > > >>> -----Original Message----- > > >>> From: gem5-dev [mailto:[email protected]] On Behalf Of Gabe > > >>> Black via gem5-dev > > >>> Sent: Tuesday, December 09, 2014 6:42 PM > > >>> To: gem5 Developer List > > >>> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) > > >>> > > >>> And... it turns out the KVM change wasn't necessary. If you're > > >>> working from my patch, get rid of where the segment limit is divided > > by PageBytes. > > >>> That was only necessary because I wasn't adding 0xFFF to the limit > > >>> when the granularity bit was set. > > >>> > > >>> Gabe > > >>> > > >>> On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black <[email protected]> > > wrote: > > >>> > > >>> > Oh, also segment limits weren't being computed correctly in the > > >>> > installSegDesc function, although I don't think that was from the > > >>> > KVM stuff. Once it was fixed it required adjusting the KVM stuff a > > >>> > little, though. > > >>> > > > >>> > Gabe > > >>> > > > >>> > On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black <[email protected]> > > >>> wrote: > > >>> > > > >>> >> Here is my patch so far. There were a few things wrong, although > > >>> >> I didn't really keep notes. The limits were mixed up, the long > > >>> >> mode bit was set on all descriptors when it's only valid for the > > >>> >> code segment, privilege level > > >>> >> 0 is the OS and 3 is for applications and not the other way > > >>> >> around, and I think the type was being set wrong for one of the > > segments. > > >>> >> Also, the syscall and sysenter registers (star and friends) > > >>> >> require the segments in the GDT to be in a particular order which > > >>> >> I don't > > >>> think they were. > > >>> >> > > >>> >> Gabe > > >>> >> > > >>> >> On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev < > > >>> >> [email protected]> wrote: > > >>> >> > > >>> >>> So, I am doing this on an AMD system and I have SE working and > > >>> >>> am able to get FS entering into virtualized mode. However, in FS > > >>> >>> I get an early exception while the kernel is booting. This seems > > >>> >>> a bit different from what Nilay and Adrian observed for FS. > > >>> >>> Could you please share the diffs that got FS working? > > >>> >>> > > >>> >>> Thanks, > > >>> >>> Alex > > >>> >>> > > >>> >>> -----Original Message----- > > >>> >>> From: gem5-dev [mailto:[email protected]] On Behalf Of > > >>> >>> Gabe Black via gem5-dev > > >>> >>> Sent: Tuesday, December 09, 2014 6:07 PM > > >>> >>> To: gem5 Developer List > > >>> >>> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) > > >>> >>> > > >>> >>> Oh, I see you have FS working again and not SE. NM, I'll keep > > >>> looking. > > >>> >>> > > >>> >>> Gabe > > >>> >>> > > >>> >>> On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black > > >>> >>> <[email protected]> > > >>> wrote: > > >>> >>> > > >>> >>> > I have FS working again which is good, but I'm still having > > >>> >>> > problems with SE. If you could let me know what you did to get > > >>> >>> > things going that would be very helpful. > > >>> >>> > > > >>> >>> > Gabe > > >>> >>> > > > >>> >>> > On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev > > >>> >>> > < [email protected]> wrote: > > >>> >>> > > > >>> >>> >> Hi Adrian, > > >>> >>> >> > > >>> >>> >> Sorry for missing your first email. I do see the interchanged > > >>> >>> >> segment limits for full system mode, though I get a different > > >>> >>> >> behaviour on my system. The simulation seems to hang in the > > >>> following manner: > > >>> >>> >> > > >>> >>> >> Processor #0 (Bootup-CPU) > > >>> >>> >> I/O APIC #1 at 0xFEC00000. > > >>> >>> >> Setting APIC routing to flat > > >>> >>> >> Processors: 1 > > >>> >>> >> PANIC: early exception rip ffffffff807909a9 error 9 cr2 > > >>> >>> >> ffffffffff5fd020 > > >>> >>> >> > > >>> >>> >> Can please provide a patch with all the modifications that > > >>> >>> >> fixed the issue on your system? > > >>> >>> >> > > >>> >>> >> Thank you, > > >>> >>> >> Alex > > >>> >>> >> ________________________________________ > > >>> >>> >> From: gem5-dev [[email protected]] on behalf of > > >>> >>> >> Adrián Colaso Diego via gem5-dev [[email protected]] > > >>> >>> >> Sent: Tuesday, December 09, 2014 2:09 AM > > >>> >>> >> To: gem5 Developer List > > >>> >>> >> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs > > >>> >>> >> Intel) > > >>> >>> >> > > >>> >>> >> You are right Nilay. I sent an email last week but nobody has > > >>> replied. > > >>> >>> >> > > >>> >>> >> It seems that descriptors (cdDesc, dsDesc and tssDesc) > > >>> >>> >> located in src/arch/x86/system.cc file are not > > >>> >>> >> well-initialized and as a consequence kvm does not work when > > running in full-system mode. > > >>> >>> >> > > >>> >>> >> Segment limits values (limitHigh and limitLow) are > > >>> >>> >> interchanged and several segment descriptor values are wrong > > >>> >>> >> too. If these values are corrected kvm works again as before. > > >>> >>> >> > > >>> >>> >> Adrian > > >>> >>> >> > > >>> >>> >> El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via > > >>> >>> >> gem5-dev > > >>> >>> escribió: > > >>> >>> >> > I also faced problem in getting KVM CPU to run in FS mode. > > >>> >>> >> > I figured > > >>> >>> >> that > > >>> >>> >> > the following changeset causes problems: > > >>> >>> >> > > > >>> >>> >> > author Alexandru Dutu <[email protected]> > > >>> >>> >> > Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) > > >>> >>> >> > changeset 10554 fe2e2f06a7c8 > > >>> >>> >> > > > >>> >>> >> > I saw the hardware reason 0x80000021, but did not try to > > >>> >>> >> > figure what was going on wrong. > > >>> >>> >> > > > >>> >>> >> > -- > > >>> >>> >> > Nilay > > >>> >>> >> > > > >>> >>> >> > On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: > > >>> >>> >> > > > >>> >>> >> > > I'm pretty sure entering 64 bit mode is the same between > > >>> >>> >> > > AMD and Intel CPUs. I vaguely remember there being some > > >>> >>> >> > > subtle page table difference though, and gem5 is building > > >>> >>> >> > > the page tables in SE mode instead of the kernel. > > >>> >>> >> > > > > >>> >>> >> > > Gabe > > >>> >>> >> > > > > >>> >>> >> > > On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via > > >>> >>> >> > > gem5-dev < [email protected]> wrote: > > >>> >>> >> > > > > >>> >>> >> > >> Hi Mike, > > >>> >>> >> > >> > > >>> >>> >> > >> trace-cmd is a very handy tool to get an overview of > > >>> >>> >> > >> what the kvm > > >>> >>> >> kernel > > >>> >>> >> > >> module is doing before going into gdb. In extreme cases > > >>> >>> >> > >> ftrace can be useful as well. > > >>> >>> >> > >> What is the error that you are seeing? Is it still > > >>> >>> >> > >> failing to enter virtualized mode? > > >>> >>> >> > >> > > >>> >>> >> > >> If that is the case and the hardware reason is > > >>> >>> >> > >> 0x80000021, that > > >>> >>> >> seems to > > >>> >>> >> > >> be an unrecoverable exception (drivers/hv/hyperv_vmbus.h > > >>> >>> >> > >> in linux > > >>> >>> >> kernel > > >>> >>> >> > >> source code). When running in SE mode, we are trying to > > >>> >>> >> > >> bring the > > >>> >>> >> machine > > >>> >>> >> > >> state to full 64bit mode without going through legacy > > modes. > > >>> >>> >> > >> It > > >>> >>> >> might be > > >>> >>> >> > >> that Intel machines have a different way of going to > > >>> >>> >> > >> 64bit mode than > > >>> >>> >> AMD > > >>> >>> >> > >> machines (different CR4, different way of enabling 64bit > > >>> >>> >> > >> mode page > > >>> >>> >> tables > > >>> >>> >> > >> etc.). I remember dealing with these issue for AMD > > >>> >>> >> > >> platforms by going through System Programming manual and > > >>> >>> >> > >> making sure > > >>> >>> >> > >> gem5 gets all the > > >>> >>> >> bits > > >>> >>> >> > >> right as there is not much the KVM kernel model will > > >>> >>> >> > >> tell about the > > >>> >>> >> cause > > >>> >>> >> > >> of failure. > > >>> >>> >> > >> > > >>> >>> >> > >> Best regards, > > >>> >>> >> > >> Alex > > >>> >>> >> > >> ________________________________________ > > >>> >>> >> > >> From: gem5-dev [[email protected]] on behalf of > > >>> >>> >> > >> Gabe Black > > >>> >>> >> via > > >>> >>> >> > >> gem5-dev [[email protected]] > > >>> >>> >> > >> Sent: Monday, December 08, 2014 7:08 PM > > >>> >>> >> > >> To: gem5 Developer List > > >>> >>> >> > >> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs > > >>> >>> >> > >> Intel) > > >>> >>> >> > >> > > >>> >>> >> > >> I'm not an expert either, but I did have problems > > >>> >>> >> > >> running KVM in SE > > >>> >>> >> mode on > > >>> >>> >> > >> an Intel CPU. I didn't look into it that much, but I > > >>> >>> >> > >> think things > > >>> >>> >> failed in > > >>> >>> >> > >> the kernel somewhere. What might be happening is that > > >>> >>> >> > >> the different > > >>> >>> >> vendors > > >>> >>> >> > >> hardware virtualization mechanisms are more or less > > >>> >>> >> > >> picky about > > >>> >>> >> various > > >>> >>> >> > >> things. Something might be set up incorrectly, and one > > >>> >>> >> implementation gets > > >>> >>> >> > >> more upset about it than the other. I believe there are > > >>> >>> >> > >> tools which > > >>> >>> >> will > > >>> >>> >> > >> help you determine whether your VM state is legal. > > >>> >>> >> > >> Perhaps Andreas > > >>> >>> >> can tell > > >>> >>> >> > >> you more about those? > > >>> >>> >> > >> > > >>> >>> >> > >> Gabe > > >>> >>> >> > >> > > >>> >>> >> > >> On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev > > >>> >>> >> > >> < > > >>> >>> >> [email protected] > > >>> >>> >> > >>> > > >>> >>> >> > >> wrote: > > >>> >>> >> > >> > > >>> >>> >> > >>> I have verified that x86 kvm works fine on AMD > > >>> >>> >> > >>> platforms, but fails > > >>> >>> >> on > > >>> >>> >> > >>> Intel platforms. > > >>> >>> >> > >>> > > >>> >>> >> > >>> Any hints about how to narrow down the cause (other > > >>> >>> >> > >>> than diving > > >>> >>> >> into gdb, > > >>> >>> >> > >>> which I will do). > > >>> >>> >> > >>> > > >>> >>> >> > >>> I am not an expert in KVM or how gem5 hooks up to > libkvm. > > >>> >>> >> > >>> _______________________________________________ > > >>> >>> >> > >>> gem5-dev mailing list > > >>> >>> >> > >>> [email protected] > > >>> >>> >> > >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > >>> > > >>> >>> >> > >> _______________________________________________ > > >>> >>> >> > >> gem5-dev mailing list > > >>> >>> >> > >> [email protected] > > >>> >>> >> > >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > >> _______________________________________________ > > >>> >>> >> > >> gem5-dev mailing list > > >>> >>> >> > >> [email protected] > > >>> >>> >> > >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > >> > > >>> >>> >> > > _______________________________________________ > > >>> >>> >> > > gem5-dev mailing list > > >>> >>> >> > > [email protected] > > >>> >>> >> > > http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > > > > >>> >>> >> > _______________________________________________ > > >>> >>> >> > gem5-dev mailing list > > >>> >>> >> > [email protected] > > >>> >>> >> > http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > > >>> >>> >> > > >>> >>> >> _______________________________________________ > > >>> >>> >> gem5-dev mailing list > > >>> >>> >> [email protected] > > >>> >>> >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> _______________________________________________ > > >>> >>> >> gem5-dev mailing list > > >>> >>> >> [email protected] > > >>> >>> >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > > >>> >>> > > > >>> >>> > > > >>> >>> _______________________________________________ > > >>> >>> gem5-dev mailing list > > >>> >>> [email protected] > > >>> >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> _______________________________________________ > > >>> >>> gem5-dev mailing list > > >>> >>> [email protected] > > >>> >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> > > >>> >> > > >>> >> > > >>> > > > >>> _______________________________________________ > > >>> gem5-dev mailing list > > >>> [email protected] > > >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> _______________________________________________ > > >>> gem5-dev mailing list > > >>> [email protected] > > >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> > > >> > > >> > > > > > _______________________________________________ > > gem5-dev mailing list > > [email protected] > > http://m5sim.org/mailman/listinfo/gem5-dev > > _______________________________________________ > > gem5-dev mailing list > > [email protected] > > http://m5sim.org/mailman/listinfo/gem5-dev > > > _______________________________________________ > gem5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/gem5-dev > _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
