Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, Jul 25, 2008 at 2:50 PM, Haavard Skinnemoen [EMAIL PROTECTED] wrote: Ok, I'll stop the chest-beating now. But please stop trying to tell people that adding a powerpc-specific option (which nobody seems to know how really works) to the command line will work on any other architectures than powerpc. You are completely right. For the purposes of this discussion, this is a powerpc-centric problem. The relocation code is written in assembly and is processor specific. Let the ARM folks make sure ARM relocation works (and I think it already does). Let the MIPS folks do the same. Each arch has it's own set of CFLAGS and LDFLAGS needed to produce a sane binary. They should be picked off and fixed one arch at a time. BTW, searching for assignments to reloc_off in the source will tell you which platforms are broken. g. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Sat, 2008-07-26 at 07:36 +0200, Wolfgang Denk wrote: In message [EMAIL PROTECTED] you wrote: I know exactly what Grant did. But he is not using the elf relocation information as that is simply not included in the data that is the u-boot.bin file. So what? Is this a problem? Why would we need to use XXX if we can perform all we need to do by using YYY instead, especially if this is simpler to implement? The only reason would be to make all arch as similar as possible. The discussion would be totally retarded if we only though about powerpc since the problem is already solved and implemented. Also this discussion is more about doing it in a way that is not powerpc specific and documented, we already have as you mentioned code for the powerpc specific way but it's not using elf relocation information. Who says that we MUST use the ELF reloc info? The post I was replaying to indicated that the existing code already did this so I simply corrected that statement. Or why? If it turns out that is is not that complicated all architectures could do it in the same way. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Sat, 2008-07-26 at 01:48 -0400, Grant Likely wrote: On Thu, Jul 24, 2008 at 2:09 PM, Kenneth Johansson [EMAIL PROTECTED] wrote: On Thu, 2008-07-24 at 10:37 -0700, vb wrote: But as you pointed out, this would work on ppc only (with a 'good' compiler), and still remains to be proven, I will get to it a bit later. I have run u-boot with everything compiled with -mrelocatable and normal relocation fixup code removed so I know it works fine. So consider it proven. But since it apparently is a problem in some situation the best is probably to make it a configuration option. Can you test against various versions of gcc? That is where I ran into trouble. Well If you have a specific version you know is not working I could test that. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Sat, Jul 26, 2008 at 3:53 AM, kenneth johansson [EMAIL PROTECTED] wrote: On Sat, 2008-07-26 at 01:48 -0400, Grant Likely wrote: On Thu, Jul 24, 2008 at 2:09 PM, Kenneth Johansson [EMAIL PROTECTED] wrote: On Thu, 2008-07-24 at 10:37 -0700, vb wrote: But as you pointed out, this would work on ppc only (with a 'good' compiler), and still remains to be proven, I will get to it a bit later. I have run u-boot with everything compiled with -mrelocatable and normal relocation fixup code removed so I know it works fine. So consider it proven. But since it apparently is a problem in some situation the best is probably to make it a configuration option. Can you test against various versions of gcc? That is where I ran into trouble. Well If you have a specific version you know is not working I could test that. gcc 4.0 worked for me. Anything newer or older seemed to behave differently. Looking at the size of the .got and .fixup sections seems to be a good indication of if things are being placed correctly. g. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: You are completely right. For the purposes of this discussion, this is a powerpc-centric problem. The relocation code is written in assembly and is processor specific. Let the ARM folks make sure ARM relocation works (and I think it already does). Let the MIPS folks do the same. Each arch has it's own set of CFLAGS and LDFLAGS needed to produce a sane binary. They should be picked off and fixed one arch at a time. Neither ARM nor MIPS do relocation. They link for a fixed address in RAM (which for example means that they cannot deal in a sane way with systems where the RAM size might change, not to mention issues with things like support for pRAM, pre-initialized framebuffers or shared log buffer which only work reasonably when you have relocation. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] This is now. Later is later. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: Or why? If it turns out that is is not that complicated all architectures could do it in the same way. That would require that all architectures implementrelocation in the same way (or at all). Actually this is not the case. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] There you go man, Keep as cool as you can. It riles them to believe that you perceive the web they weave. Keep on being free! - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On 01:57 Sat 26 Jul , Grant Likely wrote: On Fri, Jul 25, 2008 at 1:28 PM, kenneth johansson [EMAIL PROTECTED] wrote: I was afraid that what was needed was more or less a complete linker but it looks like if one generate the dynamic reloc table a much simpler linker(relocation function) is needed. Still probably a lot more complex than the GOT and fixup code that is just a loop over a table. We may need a case also ;-) The GOT relocation works. It works really well. As long as gcc and ld put the correct GOT bits into the GOT table. It seems not all versions of gcc do it in the same way with the same flags. Why not set the code depending on the GCC version? Best Regards, J. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jean-Christophe PLAGNIOL-VILLARD Sent: den 26 juli 2008 16:03 To: Grant Likely Cc: u-boot-users@lists.sourceforge.net; Wolfgang Denk; vb; Jerry Van Baren; Haavard Skinnemoen Subject: Re: [U-Boot-Users] Changing u-boot relocation scheme On 01:57 Sat 26 Jul , Grant Likely wrote: On Fri, Jul 25, 2008 at 1:28 PM, kenneth johansson [EMAIL PROTECTED] wrote: I was afraid that what was needed was more or less a complete linker but it looks like if one generate the dynamic reloc table a much simpler linker(relocation function) is needed. Still probably a lot more complex than the GOT and fixup code that is just a loop over a table. We may need a case also ;-) The GOT relocation works. It works really well. As long as gcc and ld put the correct GOT bits into the GOT table. It seems not all versions of gcc do it in the same way with the same flags. Why not set the code depending on the GCC version? For fun I had a look into eabi.asm code at http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/config/rs6000/eabi.asm?rev=1.13content-type=text/x-cvsweb-markup and I noticed one difference: The __eabi_uconvert() function skips NULL ptrs. Perhaps this is the missing piece needed in start.S for PPC? __eabi_convert pasted below for convenience. Jocke FUNC_START(__eabi_convert) cmplw 1,3,4/* any pointers to convert? */ subf5,3,4/* calculate number of words to convert */ bclr4,4/* return if no pointers */ srawi 5,5,2 addi3,3,-4/* start-4 for use with lwzu */ mtctr 5 .Lcvt: lwzu6,4(3)/* pointer to convert */ cmpwi0,6,0 beq-.Lcvt2/* if pointer is null, don't convert */ add 6,6,12/* convert pointer */ stw 6,0(3) .Lcvt2: bdnz+ .Lcvt blr - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 25 Jul 2008 22:51:09 +0200 kenneth johansson [EMAIL PROTECTED] wrote: Hmm...looks like linking with -pie (or --pic-executable) does something vaguely resembling the trick. But I don't know what vintage of ld you need for this to be available and actually work... Can't see any reason for using this flag over -fPIC for a program like u-boot. You need both. One is a compiler flag, the other is a linker flag. The linker will probably barf if you try to create a PIC executable from modules that were not compiled with -fPIC. On the other hand, using -fPIC but not -pie takes us about 95% of the way to a fully relocatable u-boot, and is what PowerPC, AVR32 and others are doing today. The remaining cases -- static and global initialized pointers -- are fixed up manually. If we link with -pie and add a bit of code to handle the dynamic relocations, we could get rid of the manual fixups. I don't know if the result would be larger or smaller than what we have today, but it would probably be a bit more robust. Does this really work without any relocation? Or is it being relocated and I'm just too blind to see it? No you are right there is probably many places like this. the code just use the old location and that works most of the time. Ah, of course. The strings are probably read directly from flash. Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Sat, 26 Jul 2008 07:36:35 +0200 Wolfgang Denk [EMAIL PROTECTED] wrote: n message [EMAIL PROTECTED] you wrote: Ok, I'll stop the chest-beating now. But please stop trying to tell people that adding a powerpc-specific option (which nobody seems to know how really works) to the command line will work on any other architectures than powerpc. OK - then please you explain exactly which other architectures have problems with relocation? Well, AVR32 doesn't exactly have a _problem_ with relocation except that it relies on a handful of manual fixups for initialized pointers. From a quick scan, Blackfin, m68k, MIPS, PPC and SPARC also rely on this. While it does work, it is somewhat fragile and requires people writing new code to know about this particular quirk if they want to use statically initialized pointers. Also, as was shown elsewhere in this thread, quite a few strings are being accessed directly from flash even after relocating to RAM. This could at least in theory interfere with the flash driver due to cache effects. So I'm a bit concerned that if PowerPC introduces its own method of relocating these data pointers, the code used on other platforms would get less exposure and therefore be more susceptible to bugs. Ideally, all platforms should do relocation the same way, and that necessarily means using standard methods like ELF dynamic relocations. Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 22:51:09 +0200 kenneth johansson [EMAIL PROTECTED] wrote: Hmm...looks like linking with -pie (or --pic-executable) does something vaguely resembling the trick. But I don't know what vintage of ld you need for this to be available and actually work... Can't see any reason for using this flag over -fPIC for a program like u-boot. You need both. One is a compiler flag, the other is a linker flag. The linker will probably barf if you try to create a PIC executable from modules that were not compiled with -fPIC. No, it won't. You just get a module with a lot more relocations to do. I have verified that all four possible combinations of the compiler -fPIC and linker -pie work and make sense. FWIW, -fPIC code on IA32 is about 16% larger than non-PIC code, while on the Blackfin, -fPIC code is about 2% larger than non-PIC code. This is an average over several large C++ applications. On the other hand, using -fPIC but not -pie takes us about 95% of the way to a fully relocatable u-boot, and is what PowerPC, AVR32 and others are doing today. The remaining cases -- static and global initialized pointers -- are fixed up manually. If we link with -pie and add a bit of code to handle the dynamic relocations, we could get rid of the manual fixups. I don't know if the result would be larger or smaller than what we have today, but it would probably be a bit more robust. I agree with this suggestion. This is the only way to ensure a sane environment, because it emulates what the compiler expects to happen. Looping over all the relocation entries and doing the right thing is architecture specific, but the process is general. The GOT entries can also be processed this way. Effort spent on this approach will tend to be more generic than the current PPC specific approach. Does this really work without any relocation? Or is it being relocated and I'm just too blind to see it? No you are right there is probably many places like this. the code just use the old location and that works most of the time. Ah, of course. The strings are probably read directly from flash. Maybe not. I have been looking at assembly dumps of short examples on the IA32 built with -fPIC. It is clear that the method of addressing static variables and static constants is DIFFERENT from the method used for global variables. The relationship of the location of the text segment (executable code), the GOT data, and the static variables/constants must remain fixed. The location of the entire program can move, but it must move in one piece. If it does move as one piece, the lea (load effective address) instructions relative to the GOT pointer will be relocated to the new address correctly. These references are based totally on the offset from the point of reference. If the code is similar on your platform (which I bet it is), then the reference will not be to the flash but rather the new place where the data was moved.. Global variables, however are referenced indirectly via 32 bit address pointers in the GOT, and these addresses must be relocated by the loader. The loader also must relocate any initialized pointers, because the program itself does not. It would be interesting to know how this is accomplished, via what relocation codes, but it does happen. Best Regards, Bill Campbell Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Sat, 26 Jul 2008 14:29:35 -0700 J. William Campbell [EMAIL PROTECTED] wrote: Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 22:51:09 +0200 kenneth johansson [EMAIL PROTECTED] wrote: Can't see any reason for using this flag over -fPIC for a program like u-boot. You need both. One is a compiler flag, the other is a linker flag. The linker will probably barf if you try to create a PIC executable from modules that were not compiled with -fPIC. No, it won't. On some platforms it will. Text relocations are nasty, so some platforms (e.g. avr32) just refuse to deal with them. But that's not really relevant -- each architecture should decide whether to compile with -fPIC or not. You just get a module with a lot more relocations to do. I have verified that all four possible combinations of the compiler -fPIC and linker -pie work and make sense. FWIW, -fPIC code on IA32 is about 16% larger than non-PIC code, while on the Blackfin, -fPIC code is about 2% larger than non-PIC code. This is an average over several large C++ applications. Right...that's counting the whole loadable image or just the .text section? Not suprising that a modern architecture like Blackfin likes -fPIC a lot better than an old beast like i386 though. I agree with this suggestion. This is the only way to ensure a sane environment, because it emulates what the compiler expects to happen. Looping over all the relocation entries and doing the right thing is architecture specific, but the process is general. The GOT entries can also be processed this way. Effort spent on this approach will tend to be more generic than the current PPC specific approach. Right...I think the GOT entries already are processed this way, sort of. Ah, of course. The strings are probably read directly from flash. Maybe not. I have been looking at assembly dumps of short examples on the IA32 built with -fPIC. It is clear that the method of addressing static variables and static constants is DIFFERENT from the method used for global variables. The relationship of the location of the text segment (executable code), the GOT data, and the static variables/constants must remain fixed. The location of the entire program can move, but it must move in one piece. If it does move as one piece, the lea (load effective address) instructions relative to the GOT pointer will be relocated to the new address correctly. These references are based totally on the offset from the point of reference. If the code is similar on your platform (which I bet it is), then the reference will not be to the flash but rather the new place where the data was moved.. Yes, address calculations in the code should be correct, as the whole thing was compiled with -fPIC. Data references, however, are usually not. The code being discussed here is an array of pointers to strings. I'm pretty sure the pointers are still pointing to flash after relocation. Global variables, however are referenced indirectly via 32 bit address pointers in the GOT, and these addresses must be relocated by the loader. The global variables themselves are accessed through the GOT, yes. But the _value_ of a global variable is currently not relocated automatically. The loader also must relocate any initialized pointers, because the program itself does not. It would be interesting to know how this is accomplished, via what relocation codes, but it does happen. This is what's currently being done manually by adding a fixed offset to all the pointers we know need to be relocated. When linking with -pie, these initialized pointers will get a dynamic relocation entry each so that we can replace all these manual fixups by simply iterating over the relocations. To summarize: Address calculations in executable code do not need to change since we already compile with -fPIC. Initialized pointers, however, are currently handled in a very suboptimal way, and linking with -pie might be one piece of the solution to this. Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Haavard Skinnemoen wrote: On Sat, 26 Jul 2008 14:29:35 -0700 J. William Campbell [EMAIL PROTECTED] wrote: Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 22:51:09 +0200 kenneth johansson [EMAIL PROTECTED] wrote: Can't see any reason for using this flag over -fPIC for a program like u-boot. You need both. One is a compiler flag, the other is a linker flag. The linker will probably barf if you try to create a PIC executable from modules that were not compiled with -fPIC. No, it won't. On some platforms it will. Text relocations are nasty, so some platforms (e.g. avr32) just refuse to deal with them. But that's not really relevant -- each architecture should decide whether to compile with -fPIC or not. You just get a module with a lot more relocations to do. I have verified that all four possible combinations of the compiler -fPIC and linker -pie work and make sense. FWIW, -fPIC code on IA32 is about 16% larger than non-PIC code, while on the Blackfin, -fPIC code is about 2% larger than non-PIC code. This is an average over several large C++ applications. Right...that's counting the whole loadable image or just the .text section? Not suprising that a modern architecture like Blackfin likes -fPIC a lot better than an old beast like i386 though. That is the entire loadable image. The percentage will therefore be slightly higher as a function of just the .text section. It is pretty representative of the effect though. On the old beast, it is not so good! I agree with this suggestion. This is the only way to ensure a sane environment, because it emulates what the compiler expects to happen. Looping over all the relocation entries and doing the right thing is architecture specific, but the process is general. The GOT entries can also be processed this way. Effort spent on this approach will tend to be more generic than the current PPC specific approach. Right...I think the GOT entries already are processed this way, sort of Ah, of course. The strings are probably read directly from flash. Maybe not. I have been looking at assembly dumps of short examples on the IA32 built with -fPIC. It is clear that the method of addressing static variables and static constants is DIFFERENT from the method used for global variables. The relationship of the location of the text segment (executable code), the GOT data, and the static variables/constants must remain fixed. The location of the entire program can move, but it must move in one piece. If it does move as one piece, the lea (load effective address) instructions relative to the GOT pointer will be relocated to the new address correctly. These references are based totally on the offset from the point of reference. If the code is similar on your platform (which I bet it is), then the reference will not be to the flash but rather the new place where the data was moved.. Yes, address calculations in the code should be correct, as the whole thing was compiled with -fPIC. Data references, however, are usually not. The code being discussed here is an array of pointers to strings. I'm pretty sure the pointers are still pointing to flash after relocation. You are correct. The contents of initialized pointers are not relocated without using the relocation data provided by -pie on the ld script. Global variables, however are referenced indirectly via 32 bit address pointers in the GOT, and these addresses must be relocated by the loader. The global variables themselves are accessed through the GOT, yes. But the _value_ of a global variable is currently not relocated automatically. The loader also must relocate any initialized pointers, because the program itself does not. It would be interesting to know how this is accomplished, via what relocation codes, but it does happen. This is what's currently being done manually by adding a fixed offset to all the pointers we know need to be relocated. When linking with -pie, these initialized pointers will get a dynamic relocation entry each so that we can replace all these manual fixups by simply iterating over the relocations. To summarize: Address calculations in executable code do not need to change since we already compile with -fPIC. Initialized pointers, however, are currently handled in a very suboptimal way, and linking with -pie might be one piece of the solution to this. I agree completely. Now we just need to add processing the relocation data. Also, the relocation vectors differ in contents from platform to platform, so a bit of custom processing will be needed for each architecture. Bill Campbell Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 25 Jul 2008 06:28:16 +0200 Wolfgang Denk [EMAIL PROTECTED] wrote: In message [EMAIL PROTECTED] you wrote: Plus it's only defined for PowerPC. What do we do on the 11 other architectures? Fix them in the first place to do reloction at all? Er. How? The only thing this thread has come up with is a powerpc-only -mrelocatable option which apparently gives the magic word to the linker to have it emit dynamic relocations. We need similar options for 11 other architectures before we can even think about starting to utilize this in a generic manner. We could build u-boot as a shared library I guess, but that feels a bit weird... Shared? Shared by what? An ELF shared library has the dynamic relocations we need. So if we build u-boot as an .so file, it should work in theory on most architectures. Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, 24 Jul 2008 19:12:20 +0200 Kenneth Johansson [EMAIL PROTECTED] wrote: We could build u-boot as a shared library I guess, but that feels a bit weird... What do you mean by that ? u-boot is already compiled with the -fPIC option. If that's sufficient, why would you need the -mrelocatable option on PowerPC? -fPIC makes the compiler generate position-independent code, which is necessary but not sufficient in order to get a fully relocatable binary. The second part of the puzzle is dynamic relocations. Shared libraries have them, and apparently, PowerPC binaries have them if you link with -mrelocatable. Currently u-boot is compiled with -fPIC, so the result is almost fully relocatable. We work around the lack of dynamic relocations by manually fixing up a handful of pointers at runtime. Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 2008-07-25 at 11:10 +0200, Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 06:28:16 +0200 Wolfgang Denk [EMAIL PROTECTED] wrote: In message [EMAIL PROTECTED] you wrote: We could build u-boot as a shared library I guess, but that feels a bit weird... Shared? Shared by what? An ELF shared library has the dynamic relocations we need. So if we build u-boot as an .so file, it should work in theory on most architectures. Haavard well the elf binary of u-boot obviously has everything we need regardless of what options it was compiled with. If we had a full linker at runtime we could just do a relink to whatever address we wanted. It sounds a bit easier to just loop over a list of pointers and change the values than to implement a complete linker but maybe that is just me. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 25 Jul 2008 13:55:58 +0200 kenneth johansson [EMAIL PROTECTED] wrote: An ELF shared library has the dynamic relocations we need. So if we build u-boot as an .so file, it should work in theory on most architectures. well the elf binary of u-boot obviously has everything we need regardless of what options it was compiled with. If we had a full linker at runtime we could just do a relink to whatever address we wanted. No we couldn't if we don't have any relocation information. Just as you can't relink just any ELF binary to run at a different location after it's been through a final link, no matter how full the linker is. And if the ELF file wasn't compiled as position-independent code to begin with, there's just no way it will ever work if it's loaded at a different address than it was linked at. You can't go through and fix up all the compiler-generated address calculations when you don't even have a clue where they are! So whether or not the u-boot ELF file has everything we need depends a _lot_ on what options were used when building it. It sounds a bit easier to just loop over a list of pointers and change the values than to implement a complete linker but maybe that is just me. The question remains how should that list of pointers be generated? One possible answer is to let the linker do it (as dynamic relocations). Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Haavard Skinnemoen Sent: den 25 juli 2008 14:19 To: kenneth johansson Cc: vb; u-boot-users@lists.sourceforge.net; Wolfgang Denk Subject: Re: [U-Boot-Users] Changing u-boot relocation scheme On Fri, 25 Jul 2008 13:55:58 +0200 kenneth johansson [EMAIL PROTECTED] wrote: An ELF shared library has the dynamic relocations we need. So if we build u-boot as an .so file, it should work in theory on most architectures. well the elf binary of u-boot obviously has everything we need regardless of what options it was compiled with. If we had a full linker at runtime we could just do a relink to whatever address we wanted. No we couldn't if we don't have any relocation information. Just as you can't relink just any ELF binary to run at a different location after it's been through a final link, no matter how full the linker is. And if the ELF file wasn't compiled as position-independent code to begin with, there's just no way it will ever work if it's loaded at a different address than it was linked at. You can't go through and fix up all the compiler-generated address calculations when you don't even have a clue where they are! So whether or not the u-boot ELF file has everything we need depends a _lot_ on what options were used when building it. Exactly. It sounds a bit easier to just loop over a list of pointers and change the values than to implement a complete linker but maybe that is just me. The question remains how should that list of pointers be generated? One possible answer is to let the linker do it (as dynamic relocations). I think the best bet is to make -mrelocatable work for PPC. Question is how does the other arches do it? Do they include fixup ptrs by default or do they lack this functionality? Jocke Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 2008-07-25 at 14:19 +0200, Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 13:55:58 +0200 kenneth johansson [EMAIL PROTECTED] wrote: An ELF shared library has the dynamic relocations we need. So if we build u-boot as an .so file, it should work in theory on most architectures. well the elf binary of u-boot obviously has everything we need regardless of what options it was compiled with. If we had a full linker at runtime we could just do a relink to whatever address we wanted. No we couldn't if we don't have any relocation information. Just as you can't relink just any ELF binary to run at a different location after it's been through a final link, no matter how full the linker is. Take time and READ what people write. I wrote compiler option and you go on about some final link that removes the relocation information. My point was that it is irrelevant if you use pic shared whatever if you are going to use the elf relocation information anyway granted you have less work to do if the object was compiled as PIC. So whether or not the u-boot ELF file has everything we need depends a _lot_ on what options were used when building it. Not really only what info you stripped way in the last linker stage. But this is going way off topic nobody is really arguing to include a linker in u-boot so it does not matter what options is available if we did. It sounds a bit easier to just loop over a list of pointers and change the values than to implement a complete linker but maybe that is just me. The question remains how should that list of pointers be generated? One possible answer is to let the linker do it (as dynamic relocations). Since I have not done a linker I probably miss some information on what makes the dynamic relocations so special ?? And could you outline how the last step in generating the binary image would work. now it works as follows. One final static link with all the .a files and a specified start address for TEXT. result is a elf file with al symbols resolved adn it can be relinked to another address if one wants but we use objcopy to make a binary. here is a patch to generate dynamic relocations in the elf file. What is the next step? objcopy -j .rela.dyn -O binary u-boot dyn_reloc_table ?? --- config.mk +++ config.mk @@ -215,7 +215,8 @@ AFLAGS := $(AFLAGS_DEBUG) -D__ASSEMBLY__ $(CPPFLAGS) -LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) +#LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) +LDFLAGS += -Bstatic -shared -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) ifneq ($(TEXT_BASE),) LDFLAGS += -Ttext $(TEXT_BASE) endif - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, Jul 25, 2008 at 7:33 AM, kenneth johansson [EMAIL PROTECTED] wrote: here is a patch to generate dynamic relocations in the elf file. What is the next step? objcopy -j .rela.dyn -O binary u-boot dyn_reloc_table ?? --- config.mk +++ config.mk @@ -215,7 +215,8 @@ AFLAGS := $(AFLAGS_DEBUG) -D__ASSEMBLY__ $(CPPFLAGS) -LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) +#LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) +LDFLAGS += -Bstatic -shared -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) ifneq ($(TEXT_BASE),) LDFLAGS += -Ttext $(TEXT_BASE) endif Interesting, but after this step we have a file which can't be converted to a runnable binary without resolving the relocation information first, right? This is not good then, as we need something to run from reset. I think this discussion covers much more ground than needed: certainly the problem of running the same executable at different addresses has been solved many times over, for different architectures and file formats. The thing is that unless we want the loader be part of u-boot we can not use those standard solutions. And I don't think we want, because it would be an unnecessary complication. I still believe that my original suggestion of determining the addresses of pointers to relocate by comparing to stripped binaries is the simplest way to do it across architectures. Apparently the same could be achieved by using -mrelocatable on powerPc, this still remains to be seen. cheers, /vb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
vb wrote: On Fri, Jul 25, 2008 at 7:33 AM, kenneth johansson [EMAIL PROTECTED] wrote: here is a patch to generate dynamic relocations in the elf file. What is the next step? objcopy -j .rela.dyn -O binary u-boot dyn_reloc_table ?? --- config.mk +++ config.mk @@ -215,7 +215,8 @@ AFLAGS := $(AFLAGS_DEBUG) -D__ASSEMBLY__ $(CPPFLAGS) -LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) +#LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) +LDFLAGS += -Bstatic -shared -T $(LDSCRIPT) $(PLATFORM_LDFLAGS) ifneq ($(TEXT_BASE),) LDFLAGS += -Ttext $(TEXT_BASE) endif Interesting, but after this step we have a file which can't be converted to a runnable binary without resolving the relocation information first, right? This is not good then, as we need something to run from reset. I think this discussion covers much more ground than needed: certainly the problem of running the same executable at different addresses has been solved many times over, for different architectures and file formats. The thing is that unless we want the loader be part of u-boot we can not use those standard solutions. And I don't think we want, because it would be an unnecessary complication. I still believe that my original suggestion of determining the addresses of pointers to relocate by comparing to stripped binaries is the simplest way to do it across architectures. Apparently the same could be achieved by using -mrelocatable on powerPc, this still remains to be seen. cheers, /vb Well, to circle back around, Grant Likely *DID* ELF-based relocation *SUCCESSFULLY* so it *CAN* be done the right way. He just didn't have a full recipe that worked over all popular compiler versions, didn't have the time to identify /why/ it didn't work on certain compiler versions, and didn't try to run on architectures that he didn't have physical access to. Doing the relocation the right way will be much more future-proof than doing it a different wrong way. The relocation information is in the ELF file until and unless we remove it. Normal ELF executables retain that relocation information... that is exactly what the L (it) is for. The linux loader (elf loader) loads an executable into an arbitrary memory location and relocates it by fixing up addresses based on the relocation table. To reiterate, Grant had a proof-of-concept (well beyond p-o-c, actually) that did everything you are asking for, but the right way and even in the same architecture that you are interested in. Despite the chest beating that has gone on in this thread about supporting other architectures, I'm 100% positive that they are also supported with fixup tables, if only we knew enough to use them. You can stand on the *shoulders* of giants or their toes. Best regards, gvb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 25 Jul 2008 16:33:56 +0200 kenneth johansson [EMAIL PROTECTED] wrote: On Fri, 2008-07-25 at 14:19 +0200, Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 13:55:58 +0200 kenneth johansson [EMAIL PROTECTED] wrote: An ELF shared library has the dynamic relocations we need. So if we build u-boot as an .so file, it should work in theory on most architectures. well the elf binary of u-boot obviously has everything we need regardless of what options it was compiled with. If we had a full linker at runtime we could just do a relink to whatever address we wanted. No we couldn't if we don't have any relocation information. Just as you can't relink just any ELF binary to run at a different location after it's been through a final link, no matter how full the linker is. Take time and READ what people write. I wrote compiler option and you go on about some final link that removes the relocation information. My point was that it is irrelevant if you use pic shared whatever if you are going to use the elf relocation information anyway granted you have less work to do if the object was compiled as PIC. Oh, you're talking about some hypothetical u-boot binary that hasn't been through the linker? How exactly do you generate it? Besides, I talked about compiler options too (in the paragraph you cut out). If you don't compile the code with -fPIC, most linkers won't be able to make the result relocatable no matter what options you specify. It sounds a bit easier to just loop over a list of pointers and change the values than to implement a complete linker but maybe that is just me. The question remains how should that list of pointers be generated? One possible answer is to let the linker do it (as dynamic relocations). Since I have not done a linker I probably miss some information on what makes the dynamic relocations so special ?? Yes, you probably do. Dynamic relocations are quite special as they are intended for the _runtime_ system, not the linker. Therefore, they are included in a loadable segment so that the program itself or the dynamic loader can go through them and perform fixups at runtime. Also, most platforms only allow a small handful of relocation types to be used as dynamic relocations. This makes the job of the dynamic loader much easier than that of the linker. And could you outline how the last step in generating the binary image would work. That's basically the question I've been trying to ask for some time. On PowerPC, I assume -mrelocatable does the trick. On other platforms, I just don't know. now it works as follows. One final static link with all the .a files and a specified start address for TEXT. result is a elf file with al symbols resolved adn it can be relinked to another address if one wants but we use objcopy to make a binary. Have you ever _tried_ to relink the final u-boot ELF file to another address? How do you do that? here is a patch to generate dynamic relocations in the elf file. What is the next step? objcopy -j .rela.dyn -O binary u-boot dyn_reloc_table ?? Might actually work, though we might need more linker options as well. At least -Bsymbolic comes to mind. And I'm not sure if -Bstatic goes well along with -shared... In any case, there's no next step. The dynamic relocations are included in a loadable segment, so they will end up in the .bin file after the last objcopy step. There will obviously be a fair amount of arch-specific code required to make the actual relocation work though. Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 2008-07-25 at 17:23 +0200, Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 16:33:56 +0200 kenneth johansson [EMAIL PROTECTED] wrote: now it works as follows. One final static link with all the .a files and a specified start address for TEXT. result is a elf file with al symbols resolved adn it can be relinked to another address if one wants but we use objcopy to make a binary. Have you ever _tried_ to relink the final u-boot ELF file to another address? How do you do that? I have before but I just did a quick test to make sure I'm not spreading miss information. ld -Ttext 0x0200 -Map u-boot_2.map -o u-boot_2 u-boot at least creates a map file that looks ok for a u-boot compiled both with -fpic and without. Did not actually run the result. here is a patch to generate dynamic relocations in the elf file. What is the next step? objcopy -j .rela.dyn -O binary u-boot dyn_reloc_table ?? Might actually work, though we might need more linker options as well. At least -Bsymbolic comes to mind. And I'm not sure if -Bstatic goes well along with -shared... In any case, there's no next step. The dynamic relocations are included in a loadable segment, so they will end up in the .bin file after the last objcopy step. hmm just checked what objcopy does and it do not work well with a object file created with shared. the resulting binary do not have .text first so it's not really workable at least not with that simple patch. cant find any option to make objcopy print out what section it actually writes out in the out file. There will obviously be a fair amount of arch-specific code required to make the actual relocation work though. So the gain of using dynamic relocation is that we have fever relocation types to implement and objcopy is not going to strip away the relocation section as it do for non dynamic relocs. still the code to actually do the relocation is architecture dependent and non existing. At least for powerpc we have a much easier option of having gcc generate a fixup table. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 16:33:56 +0200 kenneth johansson [EMAIL PROTECTED] wrote: On Fri, 2008-07-25 at 14:19 +0200, Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 13:55:58 +0200 kenneth johansson [EMAIL PROTECTED] wrote: An ELF shared library has the dynamic relocations we need. So if we build u-boot as an .so file, it should work in theory on most architectures. well the elf binary of u-boot obviously has everything we need regardless of what options it was compiled with. If we had a full linker at runtime we could just do a relink to whatever address we wanted. No we couldn't if we don't have any relocation information. Just as you can't relink just any ELF binary to run at a different location after it's been through a final link, no matter how full the linker is. Take time and READ what people write. I wrote compiler option and you go on about some final link that removes the relocation information. My point was that it is irrelevant if you use pic shared whatever if you are going to use the elf relocation information anyway granted you have less work to do if the object was compiled as PIC. Oh, you're talking about some hypothetical u-boot binary that hasn't been through the linker? How exactly do you generate it? No, he is talking about the u-boot ELF file before we strip off the relocation information in converting it to a .bin file. All ELF files contain enough relocation information to relocate the binary. If there are undefined symbols, it my not work, but it can be relocated. Besides, I talked about compiler options too (in the paragraph you cut out). If you don't compile the code with -fPIC, most linkers won't be able to make the result relocatable no matter what options you specify. The above statement is not correct. -fPIC is a compiler option requesting the compiler generate Position Independent Code. This involves adding code to relocate all references at execution time, not at load time. This makes the binary larger and slower in exchange for it running anywhere in the address space. The problem remaining with -fPIC on the PPC architecture is that certain operations still must be performed at load time. The GOT is part of the structure required to do this. It may be that we do not completely know all the things that the compiler expects to be done at load time, or it may be that with Grant's work and the correct compiler switches (-mrelocateable) we may know for the PPC. The C compiler produces relocatable code by default, but not PIC. It is true that the Linux loader can relocate ELF code on the fly as it loads into memory, PIC or not. It sounds a bit easier to just loop over a list of pointers and change the values than to implement a complete linker but maybe that is just me. The question remains how should that list of pointers be generated? One possible answer is to let the linker do it (as dynamic relocations). Since I have not done a linker I probably miss some information on what makes the dynamic relocations so special ?? Yes, you probably do. Dynamic relocations are quite special as they are intended for the _runtime_ system, not the linker. Therefore, they are included in a loadable segment so that the program itself or the dynamic loader can go through them and perform fixups at runtime. Also, most platforms only allow a small handful of relocation types to be used as dynamic relocations. This makes the job of the dynamic loader much easier than that of the linker. This is correct, the loader allows only simple relocations. Basically, the loader is doing the relocations done by the GOT if the code is PIC. The loader is not real complex. The problem is that for u-boot, the loader would need to run either at a fixed location or be itself relocated, which is a chicken and egg problem. There is a problem with assuming that -fPIC is the way to go on all architectures. First, on some computers, such as IA32 for examples, it is just not practical. There are too few registers to dedicate one register to always containing a relocation value to be added to all references dynamically. There are other examples where the concept of PIC is equally problematic for various reasons. Second, since PIC is not required for normal Linux applications, the compiler is often buggy when PIC is specified. Further, PIC is often the last feature that is totally debugged on new architectures, so even if it will eventually work, it often won't do so to begin with. Therefore, in some cases, the only approach that will work for a completely relocatable u-boot would be a small loader running in pre-assigned fixed memory locations that relocates u-boot to the desired address using the ELF relocation data. The GOT would not be available. And could you outline how the last step in generating the binary image would
Re: [U-Boot-Users] Changing u-boot relocation scheme
kenneth johansson wrote: On Fri, 2008-07-25 at 17:23 +0200, Haavard Skinnemoen wrote: [snip] There will obviously be a fair amount of arch-specific code required to make the actual relocation work though. So the gain of using dynamic relocation is that we have fever relocation types to implement and objcopy is not going to strip away the relocation section as it do for non dynamic relocs. still the code to actually do the relocation is architecture dependent and non existing. At least for powerpc we have a much easier option of having gcc generate a fixup table. EVERYTHING *already* exists, we just aren't using it right. That is what the elf loader does in a normal system. Grant did it on the PowerPC architecture - look at the archives, he didn't write new code for this, it's already in there. Here is the reversion back to the old way: http://article.gmane.org/gmane.comp.boot-loaders.u-boot/33297 If you un-revert it (re-revert it?), you will have what Grant did. For some set of PowerPC platforms, it Just Works[tm]. For other architectures and platforms, start from this and make it work. Note the list of changed files - no source file changes in it: board/cogent/u-boot.lds |1 + board/hymod/u-boot.lds |1 + board/mousse/u-boot.lds |1 + board/rsdproto/u-boot.lds |1 + cpu/mpc512x/config.mk |2 +- cpu/mpc5xx/config.mk|2 +- cpu/mpc5xx/u-boot.lds |1 + cpu/mpc5xxx/config.mk |2 +- cpu/mpc5xxx/u-boot-customlayout.lds |1 + cpu/mpc5xxx/u-boot.lds |1 + cpu/mpc8220/config.mk |2 +- cpu/mpc8220/u-boot.lds |1 + cpu/mpc824x/config.mk |2 +- cpu/mpc824x/u-boot.lds |1 + cpu/mpc8260/config.mk |2 +- cpu/mpc8260/u-boot.lds |1 + cpu/mpc83xx/config.mk |2 +- cpu/mpc83xx/u-boot.lds |1 + include/common.h|8 19 files changed, 18 insertions(+), 15 deletions(-) Best regards, gvb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 2008-07-25 at 13:02 -0400, Jerry Van Baren wrote: kenneth johansson wrote: On Fri, 2008-07-25 at 17:23 +0200, Haavard Skinnemoen wrote: [snip] There will obviously be a fair amount of arch-specific code required to make the actual relocation work though. So the gain of using dynamic relocation is that we have fever relocation types to implement and objcopy is not going to strip away the relocation section as it do for non dynamic relocs. still the code to actually do the relocation is architecture dependent and non existing. At least for powerpc we have a much easier option of having gcc generate a fixup table. EVERYTHING *already* exists, we just aren't using it right. That is what the elf loader does in a normal system. Grant did it on the PowerPC architecture - look at the archives, he didn't write new code for this, it's already in there. I know exactly what Grant did. But he is not using the elf relocation information as that is simply not included in the data that is the u-boot.bin file. Also this discussion is more about doing it in a way that is not powerpc specific and documented, we already have as you mentioned code for the powerpc specific way but it's not using elf relocation information. I was afraid that what was needed was more or less a complete linker but it looks like if one generate the dynamic reloc table a much simpler linker(relocation function) is needed. Still probably a lot more complex than the GOT and fixup code that is just a loop over a table. We may need a case also ;-) - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 25 Jul 2008 19:28:48 +0200
kenneth johansson [EMAIL PROTECTED] wrote:
I was afraid that what was needed was more or less a complete linker but
it looks like if one generate the dynamic reloc table a much simpler
linker(relocation function) is needed. Still probably a lot more complex
than the GOT and fixup code that is just a loop over a table. We may
need a case also ;-)
Hmm...looks like linking with -pie (or --pic-executable) does
something vaguely resembling the trick. But I don't know what vintage
of ld you need for this to be available and actually work...
[EMAIL PROTECTED]:~/work/u-boot/master$ avr32-linux-readelf -d u-boot
Dynamic section at offset 0x1b8f0 contains 12 entries:
TagType Name/Value
0x0004 (HASH) 0x182ac
0x0005 (STRTAB) 0x18260
0x0006 (SYMTAB) 0x18150
0x000a (STRSZ) 74 (bytes)
0x000b (SYMENT) 16 (bytes)
0x0003 (PLTGOT) 0x1ae70
0x7001 (Processor Specific: 7001) 0xe6c
0x0015 (DEBUG) 0x0
0x0007 (RELA) 0x1833c
0x0008 (RELASZ) 6996 (bytes)
0x0009 (RELAENT)12 (bytes)
0x (NULL) 0x0
The dynamic relocation section became surprisingly large. I wonder if
we actually manage to manually relocate all those buggers...I checked a
couple of them and in both cases it looked like something that needed
to be relocated, but I couldn't find any code that actually did it.
One example from common/cmd_itest.c:
op_tbl_t op_table [] = {
{ -lt, LT },
{ , LT },
{ -gt, GT },
{ , GT },
{ -eq, EQ },
{ == , EQ },
{ -ne, NE },
{ != , NE },
{ , NE },
{ -ge, GE },
{ = , GE },
{ -le, LE },
{ = , LE },
};
Does this really work without any relocation? Or is it being relocated
and I'm just too blind to see it?
The symbol tables can probably be made smaller with some careful
ldscript tweaking.
Haavard
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
U-Boot-Users mailing list
U-Boot-Users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 25 Jul 2008 11:21:12 -0400 Jerry Van Baren [EMAIL PROTECTED] wrote: The relocation information is in the ELF file until and unless we remove it. Normal ELF executables retain that relocation information... that is exactly what the L (it) is for. The linux loader (elf loader) loads an executable into an arbitrary memory location and relocates it by fixing up addresses based on the relocation table. You're mixing two different relocation tables. Statically linked executables (like u-boot) normally don't have any relocation information in them after they are fully linked. Dynamically linked executables, position-independent executables (PIE) and shared libraries have dynamic relocations which are not only in the ELF file, but in a loadable segment so that they can be accessed at run time. The relocation information from the .o files are not preserved in the final executable unless you specify the --emit-relocs flags, which exists purely for debugging purposes. The relocations used by the dynamic loader come from an entirely different, simpler set of relocation types which are normally not found in .o files. Why do you think objdump has two different options for dumping normal (-r) vs. dynamic (-R) relocations? And no, Linux ELF executables can _not_ be loaded into an arbitrary memory location unless they are PIE. Shared libraries, however, can be loaded at arbitrary memory locations, though things like pre-linking might make it more optimal to try to place them at the same address everywhere. Ok, I'll stop the chest-beating now. But please stop trying to tell people that adding a powerpc-specific option (which nobody seems to know how really works) to the command line will work on any other architectures than powerpc. Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Haavard Skinnemoen wrote: On Fri, 25 Jul 2008 11:21:12 -0400 Jerry Van Baren [EMAIL PROTECTED] wrote: The relocation information is in the ELF file until and unless we remove it. Normal ELF executables retain that relocation information... that is exactly what the L (it) is for. The linux loader (elf loader) loads an executable into an arbitrary memory location and relocates it by fixing up addresses based on the relocation table. You're mixing two different relocation tables. Statically linked executables (like u-boot) normally don't have any relocation information in them after they are fully linked. Dynamically linked executables, position-independent executables (PIE) and shared libraries have dynamic relocations which are not only in the ELF file, but in a loadable segment so that they can be accessed at run time. The relocation information from the .o files are not preserved in the final executable unless you specify the --emit-relocs flags, which exists purely for debugging purposes. The relocations used by the dynamic loader come from an entirely different, simpler set of relocation types which are normally not found in .o files. Why do you think objdump has two different options for dumping normal (-r) vs. dynamic (-R) relocations? And no, Linux ELF executables can _not_ be loaded into an arbitrary memory location unless they are PIE. Shared libraries, however, can be loaded at arbitrary memory locations, though things like pre-linking might make it more optimal to try to place them at the same address everywhere. Thanks for the explanation. I have not worked at this level with executable formats in a long time. My experience harks back to Z80 formats (I forgot which, but post-CPM which loaded into a fixed address) and .COM and .EXE formats which have rudimentary relocation fixup information so that they could be run at arbitrary addresses. Those are non-virtual (or didn't use the MMU) formats, equivalent to your PIE reference. Ok, I'll stop the chest-beating now. But please stop trying to tell people that adding a powerpc-specific option (which nobody seems to know how really works) to the command line will work on any other architectures than powerpc. Fair enough. Haavard Discovering my banner is labeled obsolete, gvb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Haavard Skinnemoen wrote:
On Fri, 25 Jul 2008 19:28:48 +0200
kenneth johansson [EMAIL PROTECTED] wrote:
I was afraid that what was needed was more or less a complete linker but
it looks like if one generate the dynamic reloc table a much simpler
linker(relocation function) is needed. Still probably a lot more complex
than the GOT and fixup code that is just a loop over a table. We may
need a case also ;-)
First of all, you are correct that fully linked executables contain no
relocation information unless the user does something like
--emit-relocs. I was wrong about that.
Hmm...looks like linking with -pie (or --pic-executable) does
something vaguely resembling the trick. But I don't know what vintage
of ld you need for this to be available and actually work...
[EMAIL PROTECTED]:~/work/u-boot/master$ avr32-linux-readelf -d u-boot
Dynamic section at offset 0x1b8f0 contains 12 entries:
TagType Name/Value
0x0004 (HASH) 0x182ac
0x0005 (STRTAB) 0x18260
0x0006 (SYMTAB) 0x18150
0x000a (STRSZ) 74 (bytes)
0x000b (SYMENT) 16 (bytes)
0x0003 (PLTGOT) 0x1ae70
0x7001 (Processor Specific: 7001) 0xe6c
0x0015 (DEBUG) 0x0
0x0007 (RELA) 0x1833c
0x0008 (RELASZ) 6996 (bytes)
0x0009 (RELAENT)12 (bytes)
0x (NULL) 0x0
The dynamic relocation section became surprisingly large. I wonder if
we actually manage to manually relocate all those buggers...I checked a
couple of them and in both cases it looked like something that needed
to be relocated, but I couldn't find any code that actually did it.
One example from common/cmd_itest.c:
op_tbl_t op_table [] = {
{ -lt, LT },
{ , LT },
{ -gt, GT },
{ , GT },
{ -eq, EQ },
{ == , EQ },
{ -ne, NE },
{ != , NE },
{ , NE },
{ -ge, GE },
{ = , GE },
{ -le, LE },
{ = , LE },
};
Does this really work without any relocation? Or is it being relocated
and I'm just too blind to see it?
Is your u-boot built with -fPIC on all the C compiles? If so, op_table
references may work without any actual relocation if the distance from
the code segment to the segment containing op_table is not changed when
u-boot is relocated to the new address. If you look at the assembly
code for op_table being used, it may be done via an offset from the
start of the code + a register pointing to the start of the code. The
offset will not change when the code is moved, and the register will
contain the correct start of code address, so it works. If it is not
-fPIC, and it is not being executed at the address for which it was
linked, I have no idea how it would work without a loader relocating
all the references. There are only 583 relocation entries specified!
I think you are correct that the --pic-executable switch is the
generic way to get the relocation addresses. I expect it is much
better supported than the PPC specific compiler switches, because ld
needs to have the relocation data on all platforms. The only additional
requirement is for ld to actually output this data. It seems pretty
likely that this is implemented on all platforms, because at least a
subset of it is required for shared libraries and dynamically linked
executables.
Best Regards,
Bill Campbell
The symbol tables can probably be made smaller with some careful
ldscript tweaking.
Haavard
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
U-Boot-Users mailing list
U-Boot-Users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/u-boot-users
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
U-Boot-Users mailing list
U-Boot-Users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, 2008-07-25 at 20:35 +0200, Haavard Skinnemoen wrote:
On Fri, 25 Jul 2008 19:28:48 +0200
kenneth johansson [EMAIL PROTECTED] wrote:
I was afraid that what was needed was more or less a complete linker but
it looks like if one generate the dynamic reloc table a much simpler
linker(relocation function) is needed. Still probably a lot more complex
than the GOT and fixup code that is just a loop over a table. We may
need a case also ;-)
Hmm...looks like linking with -pie (or --pic-executable) does
something vaguely resembling the trick. But I don't know what vintage
of ld you need for this to be available and actually work...
Can't see any reason for using this flag over -fPIC for a program like
u-boot.
[EMAIL PROTECTED]:~/work/u-boot/master$ avr32-linux-readelf -d u-boot
Dynamic section at offset 0x1b8f0 contains 12 entries:
TagType Name/Value
0x0004 (HASH) 0x182ac
0x0005 (STRTAB) 0x18260
0x0006 (SYMTAB) 0x18150
0x000a (STRSZ) 74 (bytes)
0x000b (SYMENT) 16 (bytes)
0x0003 (PLTGOT) 0x1ae70
0x7001 (Processor Specific: 7001) 0xe6c
0x0015 (DEBUG) 0x0
0x0007 (RELA) 0x1833c
0x0008 (RELASZ) 6996 (bytes)
0x0009 (RELAENT)12 (bytes)
0x (NULL) 0x0
The dynamic relocation section became surprisingly large. I wonder if
we actually manage to manually relocate all those buggers...I checked a
couple of them and in both cases it looked like something that needed
to be relocated, but I couldn't find any code that actually did it.
yes it's scary that it's working as well as it does. This is however
solved by reactivating Grant Likely's patch.
One example from common/cmd_itest.c:
op_tbl_t op_table [] = {
{ -lt, LT },
{ , LT },
{ -gt, GT },
{ , GT },
{ -eq, EQ },
{ == , EQ },
{ -ne, NE },
{ != , NE },
{ , NE },
{ -ge, GE },
{ = , GE },
{ -le, LE },
{ = , LE },
};
Does this really work without any relocation? Or is it being relocated
and I'm just too blind to see it?
No you are right there is probably many places like this. the code just
use the old location and that works most of the time.
The symbol tables can probably be made smaller with some careful
ldscript tweaking.
Haavard
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
U-Boot-Users mailing list
U-Boot-Users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: I think this discussion covers much more ground than needed: certainly the problem of running the same executable at different addresses has been solved many times over, for different architectures and file formats. The thing is that unless we want the loader be part of u-boot we can not use those standard solutions. And I don't think we want, because it would be an unnecessary complication. We do not need a loader. All we need is just relocation working completely and correctly. At the moment, it is now working completely. I still believe that my original suggestion of determining the addresses of pointers to relocate by comparing to stripped binaries is the simplest way to do it across architectures. Apparently the same Maybe I should have been more explicit right from the beginning. I repeat: forget it. This will *not* go into mainline. Zero chance. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] How many seconds are there in a year? If I tell you there are 3.155 x 10^7, you won't even try to remember it. On the other hand, who could forget that, to within half a percent, pi seconds is a nanocentury. -- Tom Duff, Bell Labs - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: I know exactly what Grant did. But he is not using the elf relocation information as that is simply not included in the data that is the u-boot.bin file. So what? Is this a problem? Why would we need to use XXX if we can perform all we need to do by using YYY instead, especially if this is simpler to implement? Also this discussion is more about doing it in a way that is not powerpc specific and documented, we already have as you mentioned code for the powerpc specific way but it's not using elf relocation information. Who says that we MUST use the ELF reloc info? Or why? I was afraid that what was needed was more or less a complete linker but it looks like if one generate the dynamic reloc table a much simpler linker(relocation function) is needed. Still probably a lot more complex than the GOT and fixup code that is just a loop over a table. We may need a case also ;-) Please re-read Grant's patch. There were NO changes to the existing simple code! Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] Murder is contrary to the laws of man and God. -- M-5 Computer, The Ultimate Computer, stardate 4731.3 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: Ok, I'll stop the chest-beating now. But please stop trying to tell people that adding a powerpc-specific option (which nobody seems to know how really works) to the command line will work on any other architectures than powerpc. OK - then please you explain exactly which other architectures have problems with relocation? Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] I am more bored than you could ever possibly be. Go back to work. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED]@transmode.se you wrote: I think the best bet is to make -mrelocatable work for PPC. Question is how does the other arches do it? Do they include fixup ptrs by default or do they lack this functionality? The question that needs to answered first is if any other architectures are affected at all, and if so, which ones. For example, ARM and MIPS (and probably most others) do *NOT* perform any relocation at all (this should be fixed, of course - being one of the big probalems we inherited from the original ARMBoot port), but as is the whole discussion is a non-isssue on these architectures. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] Systems programmers are the high priests of a low cult. -- R.S. Barton - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, Jul 24, 2008 at 2:45 AM, vb [EMAIL PROTECTED] wrote: Wolfgang, thank you for your reply, let me try to explain myself a bit clearer: On Wed, Jul 23, 2008 at 8:18 PM, Wolfgang Denk [EMAIL PROTECTED] wrote: In message [EMAIL PROTECTED] you wrote: If you invest time in solving such problems, than your time willbe much better iinvested if you try to help solving the remaining issues with Grant's code. What Grant suggests is the way to go. I do not think your approach has chances for mainline. I am sorry to hear this, especially since what I suggest would be completely compiler agnostic and would allow to avoid some of the limitations one must follow today while adding stuff to u-boot. The problem is that the u-boot C runtime setup is insane and must be fixed. Fix the C runtime and the problem goes away. The approach being suggested here would have us *preserve* the insane C runtime and actually depend on the runtime to remain insane in order to work. Not a good approach. The C runtime problem is solvable, but I didn't have the time or resources to properly dig into it and I got frustrated before it was fully debugged. g. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, Jul 24, 2008 at 2:09 PM, Kenneth Johansson [EMAIL PROTECTED] wrote: On Thu, 2008-07-24 at 10:37 -0700, vb wrote: But as you pointed out, this would work on ppc only (with a 'good' compiler), and still remains to be proven, I will get to it a bit later. I have run u-boot with everything compiled with -mrelocatable and normal relocation fixup code removed so I know it works fine. So consider it proven. But since it apparently is a problem in some situation the best is probably to make it a configuration option. Can you test against various versions of gcc? That is where I ran into trouble. g. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, Jul 25, 2008 at 10:36 PM, Wolfgang Denk [EMAIL PROTECTED] wrote: In message [EMAIL PROTECTED] you wrote: Maybe I should have been more explicit right from the beginning. I repeat: forget it. This will *not* go into mainline. Zero chance. well, it's your call, I'ill stick with -mrelocatable on 'non native' modules then. cheers, /vb Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] How many seconds are there in a year? If I tell you there are 3.155 x 10^7, you won't even try to remember it. On the other hand, who could forget that, to within half a percent, pi seconds is a nanocentury. -- Tom Duff, Bell Labs - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, Jul 25, 2008 at 10:43 PM, Grant Likely [EMAIL PROTECTED] wrote: The problem is that the u-boot C runtime setup is insane and must be fixed. Fix the C runtime and the problem goes away. The approach being suggested here would have us *preserve* the insane C runtime and actually depend on the runtime to remain insane in order to work. Grant, thank you for pitching in. It would be great if you could elaborate what exactly is insane about C runtime setup (especially since in u-boot its code comes from the source tree and not from some obscure compiler dependent library) and how its insanity is related here. TIA, /vb Not a good approach. The C runtime problem is solvable, but I didn't have the time or resources to properly dig into it and I got frustrated before it was fully debugged. g. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Fri, Jul 25, 2008 at 1:28 PM, kenneth johansson [EMAIL PROTECTED] wrote: I was afraid that what was needed was more or less a complete linker but it looks like if one generate the dynamic reloc table a much simpler linker(relocation function) is needed. Still probably a lot more complex than the GOT and fixup code that is just a loop over a table. We may need a case also ;-) The GOT relocation works. It works really well. As long as gcc and ld put the correct GOT bits into the GOT table. It seems not all versions of gcc do it in the same way with the same flags. g. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Wed, Jul 23, 2008 at 10:39:00AM -0700, vb wrote: While this relocation is seamless for u-boot in its released form, it becomes a pain each time a module needs to be added (not necessarily for following release to the Open Source community, for instance used for inhouse installations which include tens of thousand units for some companies). If these added modules were not written in position independent manner (namely, using structures with multiple stage indirect pointers interleaved with data), the effort to make these modules work in u-boot is very exhausting. Note that in U-Boot-v2 we have loadable module support; code can EXPORT_SYMBOL() public APIs to other components, which, for example, can be such custom modules. The mechanism was invented for the scenario you describe above. It comes for a cost (footprint), but the overhead is not so big as one might think. You can play around with the feature by building the sandbox target, which is a user-mode-u-boot (port of u-boot to Linux userspace). rsc -- Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de Pengutronix - Linux Solutions for Science and Industry Handelsregister: Amtsgericht Hildesheim, HRA 2686 Hannoversche Str. 2, 31134 Hildesheim, Germany Phone: +49-5121-206917-0 | Fax: +49-5121-206917-9 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Wolfgang, thank you for your reply, let me try to explain myself a bit clearer:
On Wed, Jul 23, 2008 at 8:18 PM, Wolfgang Denk [EMAIL PROTECTED] wrote:
In message [EMAIL PROTECTED] you wrote:
some companies). If these added modules were not written in position
independent manner (namely, using structures with multiple stage
indirect pointers interleaved with data), the effort to make these
modules work in u-boot is very exhausting.
I don't understand what you mean. Either you link statically with the
U-Boot image, or you use standalone programs. In both situations no
such problem as described by you exists.
we talk here about modules statically linked into the u-boot image.
Allow me to illustrate the problem I am trying to solve. Consider
adding this code to a u-boot source file on a ppc460gt platform:
vvv
87a88,105
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[]);
int (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
{
printf (pointer is %p\n, pf);
printf (function is %p\n, do_ptrt);
return 0;
}
U_BOOT_CMD(
ptrt, CFG_MAXARGS, 1, do_ptrt,
ptrt\n,
);
^^^
And this is what happens when this command is invoked:
vvv
= ptrt
pointer is fffb2754
function is 0ffb7754
So, the value of 'pf' is equal to the address of do_ptrt() *before*
relocation. The fact that there is a GOT and a sophisticated linker
script did not prevent this from happening.
Here is the suggested alternative. I will talk about recent PPC
platforms, but it could be done very similar for most of the other
platforms I believe.
I don't think so.
Admittedly I don't have experience with nearly as many different
platforms as you do, but I don't see why not, is there a specific
fundamental reason on any of the platforms? But even if this does not
work on some, it will work on most.
In general the PPC u-boot image consists of three parts: the bulk of
No, this is by far not general, not even for PPC.
I used that particular layout to emphasize that care needs to be taken
to remove the fixed address sections (the last 4k memory page in this
case) from the set of modules to link. On other architectures this
would be a different part of the image linked to the reset vector, the
idea remains the same.
Then we compare the two images and identify all locations which are
different. All these locations are 4 byte aligned, and all differ by
the same amount - equal TEXT_BASE. Now we can build a table of offsets
which need to be changed when the program is moved into a different
location, include this table in the resulting image between the bulk
and the startup page, and use this table when relocating code.
Why would we do that? The compiler already generates such a table for
free - the GOT.
well, didn't the example above show that the compiler will not do that
to an arbitrary pointer to a text object?
Basically, this implements a poor man's loader, and it will be
sufficient in case the image we are dealing with is a flat binary
(which is of course the case with u-boot). This will allow to add any
code to u-boot without much worries about being able to relocate it
properly. We would do away with storing certain structures in certain
sections, dealing with GOT, etc.
I don't see how your poor man's GOT would be any better than the
compiler generated one - except that it's horribly non-standard and
glumpsy?
Quite frankly, GOT is also clumsy and even ugly IMHO - it requires a
lot of special treatment, explicit data definitions and linker
scripts, and does not work for arbitrary data as we saw above.
I made some experiments, and this seems feasible, this could be done
as a local customization, but I would much prefer to release it to the
u-boot community and make it part of mainline - will you consider such
a patch?
I don't understand which problem you're trying to fix, and I don't see
that your solution would be any better than what we have now.
I beg to differ, the solution I am suggesting will allow to stop
worrying about placing data in special sections, and will allow to
link in third party modules which use cross segment pointers.
[snip]
because the modules are now part of u-boot, they need to be relocated
along with the rest of the image.
U-Boot does this (if statically linked) the very same way like all
other U-Boot code gets relocated.
not for pointers to functions as we saw above, right?
Why isn't it written in position independent manner?
because they come from different sources, not concerned with u-boot,
and just work and need to be ported as is.
If you want to link against U-Boot, you have
Re: [U-Boot-Users] Changing u-boot relocation scheme
vb [EMAIL PROTECTED] wrote:
int (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
{
printf (pointer is %p\n, pf);
printf (function is %p\n, do_ptrt);
return 0;
}
Just do this instead:
int (*pf)(struct cmd_tbl_s *, int, int, char *[]);
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
{
if (!pf)
pf = do_ptrt;
printf (pointer is %p\n, pf);
printf (function is %p\n, do_ptrt);
return 0;
}
IMO, it's best to avoid such pointers in the first place, especially
ones that are statically initialized.
Haavard
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
U-Boot-Users mailing list
U-Boot-Users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
vb wrote:
Wolfgang, thank you for your reply, let me try to explain myself a bit
clearer:
On Wed, Jul 23, 2008 at 8:18 PM, Wolfgang Denk [EMAIL PROTECTED] wrote:
In message [EMAIL PROTECTED] you wrote:
some companies). If these added modules were not written in position
independent manner (namely, using structures with multiple stage
indirect pointers interleaved with data), the effort to make these
modules work in u-boot is very exhausting.
I don't understand what you mean. Either you link statically with the
U-Boot image, or you use standalone programs. In both situations no
such problem as described by you exists.
we talk here about modules statically linked into the u-boot image.
Allow me to illustrate the problem I am trying to solve. Consider
adding this code to a u-boot source file on a ppc460gt platform:
vvv
87a88,105
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[]);
int (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
{
printf (pointer is %p\n, pf);
printf (function is %p\n, do_ptrt);
return 0;
}
U_BOOT_CMD(
ptrt, CFG_MAXARGS, 1, do_ptrt,
ptrt\n,
);
^^^
And this is what happens when this command is invoked:
vvv
= ptrt
pointer is fffb2754
function is 0ffb7754
So, the value of 'pf' is equal to the address of do_ptrt() *before*
relocation. The fact that there is a GOT and a sophisticated linker
script did not prevent this from happening.
OK, now I'm curious: what happens if you make the pf() pointer constant?
This will (should) change it from an initialized variable in the .data
section to a constant in the .rodata section. I'm wondering if the
.rodata section gets relocation information where the .data section doesn't.
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[]);
const int (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
{
printf (pointer is %p\n, pf);
printf (function is %p\n, do_ptrt);
return 0;
}
Thanks,
gvb
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
U-Boot-Users mailing list
U-Boot-Users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
vb wrote: What Grant suggests is the way to go. I do not think your approach has chances for mainline. I am sorry to hear this, especially since what I suggest would be completely compiler agnostic and would allow to avoid some of the limitations one must follow today while adding stuff to u-boot. cheers, /vb As an alternate approach, perhaps this is something that might be explored in the framework of the so-called U-Boot V2 effort? Specifically, if you can demonstrate it working, that would hold much more weight. You know, Show me the code.! :-) jdl - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, Jul 24, 2008 at 6:45 AM, Jon Loeliger [EMAIL PROTECTED] wrote: As an alternate approach, perhaps this is something that might be explored in the framework of the so-called U-Boot V2 effort? Specifically, if you can demonstrate it working, that would hold much more weight. You know, Show me the code.! :-) jdl now you are talking :-)! I will do something along these lines, either the compiler option or this poor man's locator thing and show you - u-boot definitely needs this enhancement. cheers, /vb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, Jul 24, 2008 at 2:58 AM, Haavard Skinnemoen [EMAIL PROTECTED] wrote: IMO, it's best to avoid such pointers in the first place, especially ones that are statically initialized. Haavard On Thu, Jul 24, 2008 at 5:23 AM, Jerry Van Baren [EMAIL PROTECTED] wrote: OK, now I'm curious: what happens if you make the pf() pointer constant? This will (should) change it from an initialized variable in the .data Haavard, Jerry, your both suggestions are very right, but you are missing an important point: I am talking about porting code to u-boot, libraries with hundreds of files, written and working in other environments - they can't be compiled in as is, and changing them is prohibitively inefficient. cheers, /vb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, 2008-07-24 at 15:01 +0200, Wolfgang Denk wrote: In message [EMAIL PROTECTED] you wrote: Quite frankly, GOT is also clumsy and even ugly IMHO - it requires a lot of special treatment, explicit data definitions and linker scripts, and does not work for arbitrary data as we saw above. It is standard, documented by the ABI, and works fine if done correctly. We just don't do it correctly yet. Best regards, Wolfgang Denk Well the type of runtime relocation that we do in u-boot is not really specified in any documentation. What is in the abi is assuming a dynamic linker and a proper elf file. Gcc can be nice and create a fixup table for us if we ask for it but that is not something that exist in any abi document I ever read. That also explains why when this reloaction code was initially done we ended up with the extra fixup code that tries to catch a few pointer that the GOT table is not containing, nobody know the fixup table even existed and it sure as hell is not mentioned in the documentation you have to read the gcc code to understand what the -mrelocatable option really do. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Kenneth Johansson [EMAIL PROTECTED] wrote: you have to read the gcc code to understand what the -mrelocatable option really do. Plus it's only defined for PowerPC. What do we do on the 11 other architectures? We could build u-boot as a shared library I guess, but that feels a bit weird... Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, 2008-07-24 at 18:57 +0200, Haavard Skinnemoen wrote: Kenneth Johansson [EMAIL PROTECTED] wrote: you have to read the gcc code to understand what the -mrelocatable option really do. Plus it's only defined for PowerPC. What do we do on the 11 other architectures? Well other architectures works in different ways and thus has other flags. We could build u-boot as a shared library I guess, but that feels a bit weird... What do you mean by that ? u-boot is already compiled with the -fPIC option. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, Jul 24, 2008 at 9:57 AM, Haavard Skinnemoen [EMAIL PROTECTED] wrote: Kenneth Johansson [EMAIL PROTECTED] wrote: you have to read the gcc code to understand what the -mrelocatable option really do. Plus it's only defined for PowerPC. What do we do on the 11 other architectures? We could build u-boot as a shared library I guess, but that feels a bit weird... I ran a quick experiment with -mrelocate, compiled a module with it and then objdump -D its contents, it seems to do exactly what I wanted - stores addresses of pointers to adjust into the fixit section. I guess the least intrusive alternative would be to compile the ported code with this option, and then add a check to the u-boot function dealing with GOT adjustments to see if the fixit section is nonempty. If so - relocate .fixit in addition to GOT. But as you pointed out, this would work on ppc only (with a 'good' compiler), and still remains to be proven, I will get to it a bit later. cheers, /vb Haavard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of vb Sent: den 24 juli 2008 20:26 To: Kenneth Johansson Cc: u-boot-users@lists.sourceforge.net; Haavard Skinnemoen; Wolfgang Denk Subject: Re: [U-Boot-Users] Changing u-boot relocation scheme On Thu, Jul 24, 2008 at 11:09 AM, Kenneth Johansson [EMAIL PROTECTED] wrote: I have run u-boot with everything compiled with -mrelocatable and normal relocation fixup code removed so I know it works fine. So consider it proven. But since it apparently is a problem in some situation the best is probably to make it a configuration option. From what I saw in my limited experiment, there is no code (as in CPU instructions) generated by gcc, just a data section containing offsets. Maybe the linker is supposed to add some code to the startup file to handle that section in case it is invoked with -mrelocatable (on't know if it would recognize this an option) or detects .fixup section in the .o files being linked. It is a mute point for embedded development anyways. I *think* the only thing missing is a simple function to scan the .fixup table and fix the pointers after relocation. You don't need that much, just reapply Grants patch(mentioned early in the debate) that were reverted because it did not work for everyone, probably due to some other strange relocation scheme. I too have run u-boot with full .fixup applied and it works. U-boot gets bigger, but that is most likely due to the old reloc code is just nullified instead of removed. Jocke - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
On Thu, 2008-07-24 at 11:26 -0700, vb wrote: On Thu, Jul 24, 2008 at 11:09 AM, Kenneth Johansson [EMAIL PROTECTED] wrote: I have run u-boot with everything compiled with -mrelocatable and normal relocation fixup code removed so I know it works fine. So consider it proven. But since it apparently is a problem in some situation the best is probably to make it a configuration option. From what I saw in my limited experiment, there is no code (as in CPU instructions) generated by gcc, just a data section containing offsets. Maybe the linker is supposed to add some code to the startup file to handle that section in case it is invoked with -mrelocatable (on't know if it would recognize this an option) or detects .fixup section in the .o files being linked. It is a mute point for embedded development anyways. I think the code exist in libgcc __eabi_convert() but code already exist in u-boot at least in several start.S files. I *think* the only thing missing is a simple function to scan the .fixup table and fix the pointers after relocation. The code should be there find . -name *.S | xargs grep FIXUP_TABLE - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Dear Kenneth, in message [EMAIL PROTECTED] you wrote: It is standard, documented by the ABI, and works fine if done correctly. We just don't do it correctly yet. Well the type of runtime relocation that we do in u-boot is not really specified in any documentation. What is in the abi is assuming a dynamic linker and a proper elf file. Gcc can be nice and create a fixup table for us if we ask for it but that is not something that exist in any abi document I ever read. That also explains why when this reloaction code was initially done we ended up with the extra fixup code that tries to catch a few pointer that the GOT table is not containing, nobody know the fixup table even existed and it sure as hell is not mentioned in the documentation you have to read the gcc code to understand what the -mrelocatable option really do. Thanks for providing such a friendly explanation that the code I came up is so crappy :-) Anyway - let's fix it, rather than do something that is *completely* non-standard. Thanks. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] Beware of the Turing Tar-pit in which everything is possible but nothing of interest is easy. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: Plus it's only defined for PowerPC. What do we do on the 11 other architectures? Fix them in the first place to do reloction at all? We could build u-boot as a shared library I guess, but that feels a bit weird... Shared? Shared by what? Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] To take a significant step forward, you must make a series of finite improvements. - Donald J. Atwood, General Motors - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
[U-Boot-Users] Changing u-boot relocation scheme
Wolfgang, I wonder if you would consider (and accept a patch) changing the u-boot architecture as described below. As we all are well aware u-boot is a great software package which speeds up bringing up new hardware immensely, its popularity is obviously growing very fast. One thing which (IMO anyway) slows down its acceptance is the way it handles relocating of itself into an arbitrary DRAM area. (Arbitrary meaning that it depends on many factors and the exact address can't be assigned ahead of time). While this relocation is seamless for u-boot in its released form, it becomes a pain each time a module needs to be added (not necessarily for following release to the Open Source community, for instance used for inhouse installations which include tens of thousand units for some companies). If these added modules were not written in position independent manner (namely, using structures with multiple stage indirect pointers interleaved with data), the effort to make these modules work in u-boot is very exhausting. Here is the suggested alternative. I will talk about recent PPC platforms, but it could be done very similar for most of the other platforms I believe. In general the PPC u-boot image consists of three parts: the bulk of code and data, the startup page (the top 4k mapped out of reset), and the reset vector (the top four bytes in the address space). The bulk is usually aligned at a certain address matching the bootprom size, and there is a gap between the bulk and the startup page. Now, let's say we link and purge the bulk section twice, once locating it at offset 0 and once at the normal u-boot TEXT_BASE, building two binary images (not including the top 4k in both cases). Then we compare the two images and identify all locations which are different. All these locations are 4 byte aligned, and all differ by the same amount - equal TEXT_BASE. Now we can build a table of offsets which need to be changed when the program is moved into a different location, include this table in the resulting image between the bulk and the startup page, and use this table when relocating code. Basically, this implements a poor man's loader, and it will be sufficient in case the image we are dealing with is a flat binary (which is of course the case with u-boot). This will allow to add any code to u-boot without much worries about being able to relocate it properly. We would do away with storing certain structures in certain sections, dealing with GOT, etc. I made some experiments, and this seems feasible, this could be done as a local customization, but I would much prefer to release it to the u-boot community and make it part of mainline - will you consider such a patch? Thank you for reading this far, please let me know what you think, Vadim - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
vb wrote: Wolfgang, I wonder if you would consider (and accept a patch) changing the u-boot architecture as described below. As we all are well aware u-boot is a great software package which speeds up bringing up new hardware immensely, its popularity is obviously growing very fast. One thing which (IMO anyway) slows down its acceptance is the way it handles relocating of itself into an arbitrary DRAM area. (Arbitrary meaning that it depends on many factors and the exact address can't be assigned ahead of time). One thing (IMO anyway) that allows u-boot to Just Work[tm] is that it relocates itself to the best location available, automatically adjusting to the probed memory configuration. There. Fixed that. ;-) While this relocation is seamless for u-boot in its released form, it becomes a pain each time a module needs to be added (not necessarily for following release to the Open Source community, for instance used for inhouse installations which include tens of thousand units for some companies). If these added modules were not written in position independent manner (namely, using structures with multiple stage indirect pointers interleaved with data), the effort to make these modules work in u-boot is very exhausting. You lost me here. What is a module? How does a it relate to u-boot? Why does it need to be relocated? Why isn't it written in position independent manner? How is your relocation methodology going to fixup a module's data structures? What pointers are in your data structures? Why do they need to be relocated? Here is the suggested alternative. I will talk about recent PPC platforms, but it could be done very similar for most of the other platforms I believe. In general the PPC u-boot image consists of three parts: the bulk of code and data, the startup page (the top 4k mapped out of reset), and the reset vector (the top four bytes in the address space). The bulk is usually aligned at a certain address matching the bootprom size, and there is a gap between the bulk and the startup page. Now, let's say we link and purge the bulk section twice, once locating it at offset 0 and once at the normal u-boot TEXT_BASE, building two binary images (not including the top 4k in both cases). Then we compare the two images and identify all locations which are different. All these locations are 4 byte aligned, and all differ by the same amount - equal TEXT_BASE. Now we can build a table of offsets which need to be changed when the program is moved into a different location, include this table in the resulting image between the bulk and the startup page, and use this table when relocating code. Gcc supports proper relocation, if only we knew how to make it work for all reasonable versions of gcc (and Grant's conclusion is that many versions of gcc in use today do *not* support the relocation). http://article.gmane.org/gmane.comp.boot-loaders.u-boot/36343/ Grant Likely created a patch that did this in the 1.3.0 timeframe, but ended up reverting it due to toolset problems - it was a patch before its time. To read up on the history, follow this search: http://search.gmane.org/?query=relocationauthor=grant+likelygroup=gmane.comp.boot-loaders.u-bootsort=dateDEFAULTOP=andxP=Zreloc%09Zchang%09Zgrant%09ZlikexFILTERS=Gcomp.boot-loaders.u-boot---A Basically, this implements a poor man's loader, and it will be sufficient in case the image we are dealing with is a flat binary (which is of course the case with u-boot). This will allow to add any code to u-boot without much worries about being able to relocate it properly. We would do away with storing certain structures in certain sections, dealing with GOT, etc. I made some experiments, and this seems feasible, this could be done as a local customization, but I would much prefer to release it to the u-boot community and make it part of mainline - will you consider such a patch? Thank you for reading this far, please let me know what you think, On one hand, your proposal sounds fairly simple and independent of the gcc version. The primary cost would be extra build steps 1) Link at TEXT_BASE 2) Link at address 0 3) Binary diff the two to find addresses that need relocation 4) Rebuild with the diff table (compile the diff table and relink) On the other hand, the skeptic in me says it ain't that easy. Go ahead, prove me wrong. ;-) Vadim Best regards, gvb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
Jerry, thank you for your reply, please see below: On Wed, Jul 23, 2008 at 2:46 PM, Jerry Van Baren [EMAIL PROTECTED] wrote: vb wrote: One thing which (IMO anyway) slows down its acceptance is the way it handles relocating of itself into an arbitrary DRAM area. (Arbitrary meaning that it depends on many factors and the exact address can't be assigned ahead of time). One thing (IMO anyway) that allows u-boot to Just Work[tm] is that it relocates itself to the best location available, automatically adjusting to the probed memory configuration. There. Fixed that. ;-) Sorry, I was not clear - it indeed is a great property of u-boot that it can run on any probed memory config. What I meant to say was that the way it relocates itself is not perfect, and this is why u-boot is not becoming more ubiquitous. While this relocation is seamless for u-boot in its released form, it becomes a pain each time a module needs to be added (not necessarily for following release to the Open Source community, for instance used for inhouse installations which include tens of thousand units for some companies). If these added modules were not written in position independent manner (namely, using structures with multiple stage indirect pointers interleaved with data), the effort to make these modules work in u-boot is very exhausting. You lost me here. What is a module? for the purposes of this discussion any .o file or any library of .o files. How does a it relate to u-boot? I need to extend u-boot with certain functionality available from those modules. Why does it need to be relocated? because the modules are now part of u-boot, they need to be relocated along with the rest of the image. Why isn't it written in position independent manner? because they come from different sources, not concerned with u-boot, and just work and need to be ported as is. How is your relocation methodology going to fixup a module's data structures? The relocation will adjust only pointers to absolute addresses stored in text/data segments. What pointers are in your data structures? Why do they need to be relocated? in some cases data structures include pointers to other data structures, etc. (This is called 'tree' btw, sorry couldn't resist :-) [snip] Gcc supports proper relocation, if only we knew how to make it work for all reasonable versions of gcc (and Grant's conclusion is that many versions of gcc in use today do *not* support the relocation). http://article.gmane.org/gmane.comp.boot-loaders.u-boot/36343/ well, if there is a pointer to a data element or to a function stored in a data structure, the .bin image is built and fixed using actual addresses set by the linker. How can gcc help here? Grant Likely created a patch that did this in the 1.3.0 timeframe, but ended up reverting it due to toolset problems - it was a patch before its time. To read up on the history, follow this search: http://search.gmane.org/?query=relocationauthor=grant+likelygroup=gmane.comp.boot-loaders.u-bootsort=dateDEFAULTOP=andxP=Zreloc%09Zchang%09Zgrant%09ZlikexFILTERS=Gcomp.boot-loaders.u-boot---A I will read up on this. [snip] I made some experiments, and this seems feasible, this could be done as a local customization, but I would much prefer to release it to the u-boot community and make it part of mainline - will you consider such a patch? Thank you for reading this far, please let me know what you think, On one hand, your proposal sounds fairly simple and independent of the gcc version. The primary cost would be extra build steps 1) Link at TEXT_BASE 2) Link at address 0 3) Binary diff the two to find addresses that need relocation 4) Rebuild with the diff table (compile the diff table and relink) add here inserting code to process the table and modify pointers after relocation (which is already being done, but suing a different algorithm) On the other hand, the skeptic in me says it ain't that easy. Go ahead, prove me wrong. ;-) I am sure it is doable, I am just trying to understand if this would be accepted in general, because it can be done differently depending if one needs to push it upstream or not. cheers, /vb Vadim Best regards, gvb - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: I wonder if you would consider (and accept a patch) changing the u-boot architecture as described below. Consider: always. Accept: if it makes more sense than the current implementation. As we all are well aware u-boot is a great software package which speeds up bringing up new hardware immensely, its popularity is obviously growing very fast. One thing which (IMO anyway) slows down its acceptance is the way it handles relocating of itself into an arbitrary DRAM area. (Arbitrary meaning that it depends on many factors and the exact address can't be assigned ahead of time). The address is not arbitrary, and in fact it CANNOT be determined in advance, especially on boards with varying RAM sized (for example using standard DIMMs). Since U-Boot will relocate itself normally to the end of the available RAM, you cannot know the relocation address in advance. Also, for example things like the PRAM size can be set by the user through environment variables - again this will cause the relocation address to become variable. While this relocation is seamless for u-boot in its released form, it becomes a pain each time a module needs to be added (not necessarily for following release to the Open Source community, for instance used for inhouse installations which include tens of thousand units for some companies). If these added modules were not written in position independent manner (namely, using structures with multiple stage indirect pointers interleaved with data), the effort to make these modules work in u-boot is very exhausting. I don't understand what you mean. Either you link statically with the U-Boot image, or you use standalone programs. In both situations no such problem as described by you exists. Here is the suggested alternative. I will talk about recent PPC platforms, but it could be done very similar for most of the other platforms I believe. I don't think so. In general the PPC u-boot image consists of three parts: the bulk of No, this is by far not general, not even for PPC. Now, let's say we link and purge the bulk section twice, once locating it at offset 0 and once at the normal u-boot TEXT_BASE, building two binary images (not including the top 4k in both cases). Then we compare the two images and identify all locations which are different. All these locations are 4 byte aligned, and all differ by the same amount - equal TEXT_BASE. Now we can build a table of offsets which need to be changed when the program is moved into a different location, include this table in the resulting image between the bulk and the startup page, and use this table when relocating code. Why would we do that? The compiler already generates such a table for free - the GOT. Basically, this implements a poor man's loader, and it will be sufficient in case the image we are dealing with is a flat binary (which is of course the case with u-boot). This will allow to add any code to u-boot without much worries about being able to relocate it properly. We would do away with storing certain structures in certain sections, dealing with GOT, etc. I don't see how your poor man's GOT would be any better than the compiler generated one - except that it's horribly non-standard and glumpsy? I made some experiments, and this seems feasible, this could be done as a local customization, but I would much prefer to release it to the u-boot community and make it part of mainline - will you consider such a patch? I don't understand which problem you're trying to fix, and I don't see that your solution would be any better than what we have now. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] I object to intellect without discipline; I object to power without constructive purpose. -- Spock, The Squire of Gothos, stardate 2124.5 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
Re: [U-Boot-Users] Changing u-boot relocation scheme
In message [EMAIL PROTECTED] you wrote: What is a module? for the purposes of this discussion any .o file or any library of .o files. Neither of these is executable - you either have to link these with the U-Boot code or into a standalone application - and in both cases none of the problems you describe happens. How does a it relate to u-boot? I need to extend u-boot with certain functionality available from those modules. OK - as mentioned: either statically linked or as standalone app. Why does it need to be relocated? because the modules are now part of u-boot, they need to be relocated along with the rest of the image. U-Boot does this (if statically linked) the very same way like all other U-Boot code gets relocated. Why isn't it written in position independent manner? because they come from different sources, not concerned with u-boot, and just work and need to be ported as is. If you want to link against U-Boot, you have to use at least compatible compiler options and the U-Boot linker script. How is your relocation methodology going to fixup a module's data structures? The relocation will adjust only pointers to absolute addresses stored in text/data segments. No. What pointers are in your data structures? Why do they need to be relocated? in some cases data structures include pointers to other data structures, etc. (This is called 'tree' btw, sorry couldn't resist :-) This should get sorted out correctly when linking. Gcc supports proper relocation, if only we knew how to make it work for all reasonable versions of gcc (and Grant's conclusion is that many versions of gcc in use today do *not* support the relocation). http://article.gmane.org/gmane.comp.boot-loaders.u-boot/36343/ well, if there is a pointer to a data element or to a function stored in a data structure, the .bin image is built and fixed using actual addresses set by the linker. How can gcc help here? By adding the respective entries to the GOT. I am sure it is doable, I am just trying to understand if this would be accepted in general, because it can be done differently depending if one needs to push it upstream or not. If you invest time in solving such problems, than your time willbe much better iinvested if you try to help solving the remaining issues with Grant's code. What Grant suggests is the way to go. I do not think your approach has chances for mainline. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] Nothing in progression can rest on its original plan. We may as well think of rocking a grown man in the cradle of an infant. - Edmund Burke - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ U-Boot-Users mailing list U-Boot-Users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/u-boot-users
