Calculating virtual address from physical address
Chris Dumoulin wrote: Thanks for your reply; I found it very useful and interesting. Now, I have a whole bunch of questions. You said that the temporary TLB entries setup in head_4xx.S will eventually be replaced. Where is the code that creates these new TLB entries later on? Are the 'real' TLB entries only created once, and persist for as long as the system is running, or do TLB entries change often while the system is running? It has been a few months since I was deep in this, so I am weaker on details at the moment. But the gist is that the MMU in PowerPC's is primarily software driven. It functions as a cache - there are alot of details, but unless you arfe getting really deep into memory management you can think of the MMU as a 64 entry cache. Software - in this instance the Linux VM system is responsible for deciding exactly what happens when the cache is full and a new entry needs added. Manually stuffing an entry into the MMU is safe up until that event occurs. The VM system entries (real entries if you wish) are in Linux Memory management data structures - page tables etc. When a page fault occurs Linux looks up the correct entry in its tables and replaces one in the MMU with the required one. Unlike X86's where much of this is implimented in hardware, in a PowerPC the replacement algorithm can be anything you want - it is written in software. Therefore handling page faults is likely to be slower, but the OS is in total control of all aspects of Memory management. It has very few constraints imposed on it by the MMU. Real entries are created and destroyed inside the kernel by anything that wants memory. Drivers demand mapping of IO based memory typically when they initiallize and should release it when they unload. Programs request memory when they need it and release it when they are done. There are subtle differences between IO memory mapping - the virtual address for an IO mapped memory device MUST corresponf to a specific set of addresses, while ordinary requests for a memory mapping can be satisfied by most any block of memory. Do you know what state the MSR will be in at this point in the code? I know what the power-on reset state is, but I'm wondering if it'll be in a different state by the time we get to this point in head_4xx.S. I am not sure that Linux sets the MSR at any point prior to head_4xx.S. Regardless, greping the ppc directories within kernel source for MSR_KERNEL will expose the bits their definitions and the normal state, In my instance to avoid machine checks I had to conditionally redefine MSR_KERNEL in one header file to avoid machine checks. When you suggest disabling instruction or data address translation, is that just so I could access my hardware directly, or is there some other reason? Atleast for me getting through the rfi to start_here: which should be where you end up immediately after the rfi proved very difficult. by enabling bits one at a time I was able to test what was happening and establish what was working. I.E if you only enable instruction translation, you can still write to your physical IO port, but the 'rfi' will take you to the virtual address 'start_here:' This was solely a debugging and problem isolation approach. It also enabled me to test things bit by bit and assure my self that everything worked, while loading the MSR with the default KERNEL MSR value and executing the rfi presumes that a number of things are all setup properly - a failure in any of them would create a problem. It is not often in programming that a single instructions makes so many changes all at once, and therefore in one instant requires so many of them to be right. I actually wrote some code the stuffed a value at specific physical addresses, turned on data address mapping, read the value from the correct virtual address turned of mapping and then wrote the value to my debug port. I also was able to test the TLB entry I inserted the same way. The bit by bit approach is just a way to figure out why you can not get from real mode to virtual mode by dividing the problem into small testable peices. You were enabling the MSR bits, one at a time, and found that the machine check was causing the hang (I'm assuming that's what you meant by 'sent me to space'). Was the idea there to just isolate what type of exception was causing the hang, or were you looking to make some permanent changes to the MSR? Is a machine check interrupt caused by trying to access an address that doesn't have a TLB entry? Unless I am completely mistaken, machine checks are not cause by softwded are or programming errors, they are cause by hardware problems, or atleast by hardware reporting problems In my instance I forwarded I c the problem to the FPGA programmers, and disabled the machine check so that I could move on. I was able to get some clues prior to my bit by bit tests.
Calculating virtual address from physical address
On Wed, May 10, 2006 at 07:04:30AM -0400, David H. Lynch Jr. wrote: [skip] Unless I am completely mistaken, machine checks are not cause by softwded are or programming errors, they are cause by hardware problems, You can trivially trigger MachineChecks without any hardware problems - just map some non-existent physical address and try to access it. -- Eugene
Calculating virtual address from physical address
How is a virtual address calculated for a high address like 0x4F60? Add this to the MMU initialzation code (initial_mmu): /* map leds - 1:1 phys=virt */ lis r3,0x4F60 at h ori r3,r3,0x4F60 at l mr r4,r3 clrrwi r4,r4,12 ori r4,r4,(TLB_WR|TLB_I|TLB_M|TLB_G) clrrwi r3,r3,12 ori r3,r3,(TLB_VALID | TLB_PAGESZ(PAGESZ_4K)) li r0,0/* TLB slot 0 */ tlbwe r4,r0,TLB_DATA tlbwe r3,r0,TLB_TAG Then you have a one to one mapping between physical and virtual memory, and you can access the LEDs at the same address after turning one the MMU. BTW, we had a similar problem, but the reason was not turning on the MMU. The MMU is enabled by writing MSR_KERNEL the status register. This also enables the machine state exception interrupt. Our hardware design produced PLB bus exceptions, which was unnoticed on standalone applications. But this bus exceptions also trigger machine state exceptions. So instead of jumping to start_here the processor branched to 0x200 - unfortunatly the trap table is not set up at this point in the boot process... Alex
Calculating virtual address from physical address
Thanks for your reply; I found it very useful and interesting. Now, I have a whole bunch of questions. You said that the temporary TLB entries setup in head_4xx.S will eventually be replaced. Where is the code that creates these new TLB entries later on? Are the 'real' TLB entries only created once, and persist for as long as the system is running, or do TLB entries change often while the system is running? Do you know what state the MSR will be in at this point in the code? I know what the power-on reset state is, but I'm wondering if it'll be in a different state by the time we get to this point in head_4xx.S. When you suggest disabling instruction or data address translation, is that just so I could access my hardware directly, or is there some other reason? You were enabling the MSR bits, one at a time, and found that the machine check was causing the hang (I'm assuming that's what you meant by 'sent me to space'). Was the idea there to just isolate what type of exception was causing the hang, or were you looking to make some permanent changes to the MSR? Is a machine check interrupt caused by trying to access an address that doesn't have a TLB entry? Can you point me to some information about Grant's platform bus changes that you were talking about? I am using a custom V2Pro board, and I'd be interested to see if this code is something I should be looking at. Thanks alot, Chris On May 05, David H. Lynch Jr. dhlii at dlasys.net wrote: Chris Dumoulin wrote: My LEDs are at address 0x4F60 and my CONFIG_KERNEL_START is 0xC000. If this address were low enough, I would just add 0xC000 to the address to get the virtual address, but since my LED address is so high, the sum will be well past the 32-bit maximum address value. How is a virtual address calculated for a high address like 0x4F60? There are macros tophys and tovirt that convert addresses between physical and virtual. There are use example in the head_4xx.S file you are already in. If you are going to use a port for debugging you need to create a tlb entry for it. Same file in initial_mmu the code inside the if defined(CONFIG_SERIAL_TEXT_DEBUG) should provide an example how to do that. Be forwarned that any entries you create now will eventually disappear (took 2 weeks to figure that out once), but that may not happen intil after /init starts. Also with a little of jiggering arround the bits in MSR_KERNEL you can enable Data address translation independently of instruction address translation as well as disable or enable a variety of checks. It took me three weeks to get a new Xilinx V4 board through the rfi and to start_here in the same turn_on_mmu code you are working on. Eventually I ended up enabling the MSR bits one at a time until I discovered that enabling the Machine Check sent me to space. Regardless, once I relialized I could test the code with the MSR bits enabled one at a time isolating the problem became easier. The two issues I addressed above which relate specifically to your efforts with the ml300, constituted more than 80% of my effort to get a Xilinx Virtex 4 running. Finally, I started prior to grants platform bus changes. I have been adapting my V4 code to fit with Grants changes (the client has what they want so they do not care) I have not put alot of effort into this, but I currently get waylayed much later in new platform bus specific initialization code. I had no problem with the older board specific initialization code. If you are running on a real ml300 I am surprised you are having any problems though I do not have an ml300 to check that. But if you are running on a custom V2Pro board you have to get the board specific initalization right and therefore could trip over the issue I am currently having migrating from old to new. BTW, he is the assembly code that I'm working with (from arch/ppc/kernel/head_4xx.S): .text _GLOBAL(_stext) _GLOBAL(_start) /* Save parameters we are passed. */ mrr31,r3 mrr30,r4 mrr29,r5 mrr28,r6 mrr27,r7 /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_0 stw r25,0(r26) /* We have to turn on the MMU right away so we get cache modes * set correctly. */ blinitial_mmu /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_1 stw r25,0(r26) /* We now have the lower 16 Meg mapped into TLB entries, and the caches * ready to work. */ turn_on_mmu: lisr0,MSR_KERNEL at h orir0,r0,MSR_KERNEL at l mtsprSPRN_SRR1,r0 lisr0,start_here at h orir0,r0,start_here at l mtspr
Calculating virtual address from physical address
On 5/6/06, Chris Dumoulin cdumoulin at ics-ltd.com wrote: You said that the temporary TLB entries setup in head_4xx.S will eventually be replaced. Where is the code that creates these new TLB entries later on? Are the 'real' TLB entries only created once, and persist for as long as the system is running, or do TLB entries change often while the system is running? The kernel maintains a list of mappings between virtual and physical space. When the processor takes a TLB miss exception, then exception handler loads the needed mapping into the TLB and returns from exception. (Look in head_4xx.S; specifically at finish_tlb_load). TLB entries are loaded in a round-robin fashion as needed. Since your early TLB mappings aren't in the kernel page tables; they get overwritten and can't be reloaded by the TLB miss exception handler. Can you point me to some information about Grant's platform bus changes that you were talking about? I am using a custom V2Pro board, and I'd be interested to see if this code is something I should be looking at. The platform bus changes moves the devices to use the Platform Bus infrastructure. It's kind of a move away from multiple processor-specific bus management schemes for simple busses. ie. If the processor can access it without special setup code; the device can go on the platform bus. Check out http://patchwork.ozlabs.org/linuxppc/ and search for my name. There's about 9 patches grouped together. Cheers, g. -- Grant Likely, B.Sc. P.Eng. Secret Lab Technologies Ltd. (403) 399-0195
Calculating virtual address from physical address
I'm using a Virtex II Pro-based board with a PPC405. The board is hanging somewhere very early in the kernel boot process. I believe it may be dying at the point where the MMU is enabled. In order to determine the exact point at which my board hangs, I'm blinking two LEDs in the assembly code found in arch/ppc/kernel/head_4xx.S, . Currently I am only able to successfully access the LEDs before the MMU is turned on, but I can't be sure that I'm calculating the virtual address properly when I try to access the LED after the MMU is turned on. My LEDs are at address 0x4F60 and my CONFIG_KERNEL_START is 0xC000. If this address were low enough, I would just add 0xC000 to the address to get the virtual address, but since my LED address is so high, the sum will be well past the 32-bit maximum address value. How is a virtual address calculated for a high address like 0x4F60? BTW, he is the assembly code that I'm working with (from arch/ppc/kernel/head_4xx.S): .text _GLOBAL(_stext) _GLOBAL(_start) /* Save parameters we are passed. */ mrr31,r3 mrr30,r4 mrr29,r5 mrr28,r6 mrr27,r7 /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_0 stw r25,0(r26) /* We have to turn on the MMU right away so we get cache modes * set correctly. */ blinitial_mmu /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_1 stw r25,0(r26) /* We now have the lower 16 Meg mapped into TLB entries, and the caches * ready to work. */ turn_on_mmu: lisr0,MSR_KERNEL at h orir0,r0,MSR_KERNEL at l mtsprSPRN_SRR1,r0 lisr0,start_here at h orir0,r0,start_here at l mtsprSPRN_SRR0,r0 SYNC /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_2 stw r25,0(r26) rfi/* enables MMU */ /* CRD: set LED state here */ /* This address should be a virtual address */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_3 stw r25,0(r26) b./* prevent prefetch past rfi */ Regards, Chris Dumoulin -- *--Christopher Dumoulin--* Software Team Leader http://ics-ltd.com/ http://ics-ltd.com/ Interactive Circuits and Systems Ltd. 5430 Canotek Road Ottawa, ON K1J 9G2 (613)749-9241 1-800-267-9794 (USA only) This e-mail is private and confidential and is for the addressee only. If misdirected, please notify us by telephone and confirm that it has been deleted from your system and any hard copies destroyed. You are strictly prohibited from using, printing, distributing or disseminating it or any information contained in it save to the intended recipient.
Calculating virtual address from physical address
Chris Dumoulin wrote: I'm using a Virtex II Pro-based board with a PPC405. The board is hanging somewhere very early in the kernel boot process. I believe it may be dying at the point where the MMU is enabled. In order to determine the exact point at which my board hangs, I'm blinking two LEDs in the assembly code found in arch/ppc/kernel/head_4xx.S, . Currently I am only able to successfully access the LEDs before the MMU is turned on, but I can't be sure that I'm calculating the virtual address properly when I try to access the LED after the MMU is turned on. Typical when trying to bring up board ... Once the MMU is turned on, you leds register are most likely ... nowhere ... i.e. if you don't create a mapping your self there is just no virtual address that will access your leds physical address. What I did on some ppc work was tu use a quick BAT mapping to map some leds. It's pretty easy to setup. Be aware though that this mapping will get wiped out when the kernel sets up the BAT for itself. Sylvain
Calculating virtual address from physical address
On Fri, May 05, 2006 at 09:27:50PM +0200, Sylvain Munaut wrote: Chris Dumoulin wrote: I'm using a Virtex II Pro-based board with a PPC405. The board is hanging somewhere very early in the kernel boot process. I believe it may be dying at the point where the MMU is enabled. In order to determine the exact point at which my board hangs, I'm blinking two LEDs in the assembly code found in arch/ppc/kernel/head_4xx.S, . Currently I am only able to successfully access the LEDs before the MMU is turned on, but I can't be sure that I'm calculating the virtual address properly when I try to access the LED after the MMU is turned on. Typical when trying to bring up board ... Once the MMU is turned on, you leds register are most likely ... nowhere ... i.e. if you don't create a mapping your self there is just no virtual address that will access your leds physical address. What I did on some ppc work was tu use a quick BAT mapping to map some leds. It's pretty easy to setup. Be aware though that this mapping will get wiped out when the kernel sets up the BAT for itself. There are no BATs on 4xx. However, the same conceptual thing can be done by wiring a fixed TLB entry to cover those LEDs temporarily during bringup debug. The temporary TLB entry will be wiped out by normal tlb misses after things are running whenever the fixed entry is clobbered by the round robin replacement. -Matt
Calculating virtual address from physical address
On Fri, 2006-05-05 at 13:24 -0700, Matt Porter wrote: There are no BATs on 4xx. However, the same conceptual thing can be done by wiring a fixed TLB entry to cover those LEDs temporarily during bringup debug. The temporary TLB entry will be wiped out by normal tlb misses after things are running whenever the fixed entry is clobbered by the round robin replacement. At which point you should be able to call ioremap to generate a virtual address inside a device driver or board file. josh
Calculating virtual address from physical address
Chris Dumoulin wrote: My LEDs are at address 0x4F60 and my CONFIG_KERNEL_START is 0xC000. If this address were low enough, I would just add 0xC000 to the address to get the virtual address, but since my LED address is so high, the sum will be well past the 32-bit maximum address value. How is a virtual address calculated for a high address like 0x4F60? There are macros tophys and tovirt that convert addresses between physical and virtual. There are use example in the head_4xx.S file you are already in. If you are going to use a port for debugging you need to create a tlb entry for it. Same file in initial_mmu the code inside the if defined(CONFIG_SERIAL_TEXT_DEBUG) should provide an example how to do that. Be forwarned that any entries you create now will eventually disappear (took 2 weeks to figure that out once), but that may not happen intil after /init starts. Also with a little of jiggering arround the bits in MSR_KERNEL you can enable Data address translation independently of instruction address translation as well as disable or enable a variety of checks. It took me three weeks to get a new Xilinx V4 board through the rfi and to start_here in the same turn_on_mmu code you are working on. Eventually I ended up enabling the MSR bits one at a time until I discovered that enabling the Machine Check sent me to space. Regardless, once I relialized I could test the code with the MSR bits enabled one at a time isolating the problem became easier. The two issues I addressed above which relate specifically to your efforts with the ml300, constituted more than 80% of my effort to get a Xilinx Virtex 4 running. Finally, I started prior to grants platform bus changes. I have been adapting my V4 code to fit with Grants changes (the client has what they want so they do not care) I have not put alot of effort into this, but I currently get waylayed much later in new platform bus specific initialization code. I had no problem with the older board specific initialization code. If you are running on a real ml300 I am surprised you are having any problems though I do not have an ml300 to check that. But if you are running on a custom V2Pro board you have to get the board specific initalization right and therefore could trip over the issue I am currently having migrating from old to new. BTW, he is the assembly code that I'm working with (from arch/ppc/kernel/head_4xx.S): .text _GLOBAL(_stext) _GLOBAL(_start) /* Save parameters we are passed. */ mrr31,r3 mrr30,r4 mrr29,r5 mrr28,r6 mrr27,r7 /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_0 stw r25,0(r26) /* We have to turn on the MMU right away so we get cache modes * set correctly. */ blinitial_mmu /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_1 stw r25,0(r26) /* We now have the lower 16 Meg mapped into TLB entries, and the caches * ready to work. */ turn_on_mmu: lisr0,MSR_KERNEL at h orir0,r0,MSR_KERNEL at l mtsprSPRN_SRR1,r0 lisr0,start_here at h orir0,r0,start_here at l mtsprSPRN_SRR0,r0 SYNC /* CRD: set LED state here */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_2 stw r25,0(r26) rfi/* enables MMU */ /* CRD: set LED state here */ /* This address should be a virtual address */ lisr26,0x4F60 at h orir26,r26,0x4F60 at l li r25,LED_STATE_3 stw r25,0(r26) b./* prevent prefetch past rfi */ Regards, Chris Dumoulin -- Dave Lynch DLA Systems Software Development:Embedded Linux 717.627.3770 dhlii at dlasys.nethttp://www.dlasys.net fax: 1.253.369.9244Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction. Albert Einstein
