Sorry Gabe, forgot to address your other question. We have an internal set of "unusual" binaries, which are testing architectural features.
Some of these tests mark pages with different address access probably to check accesses which are crossing page boundaries, so they are built using 1 segment per page. Giacomo ________________________________ From: gem5-dev <[email protected]> on behalf of Giacomo Travaglini <[email protected]> Sent: 22 October 2019 10:44 To: Gabe Black <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: [gem5-dev] Elf loading problem Right I will push a new version where the segments are named after the index. This is in any case how they are called when you parse the elf binary with the readelf --segments. If someone wants to know which sections a segment contains, could just cross reference the segment name in gem5 with the readelf --segments output ________________________________ From: Gabe Black <[email protected]> Sent: 22 October 2019 00:48 To: Giacomo Travaglini <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: Elf loading problem There are some problems with that patch which I described in the review. To circle back, why does your ELF file have such a crazy number of segments and sections? That seems highly unusual, and it would be nice not to have to go to extraordinary lengths to support such an atypical input. I would say if it really is prohibitive to figure out names based on sections, to just call the segments segment 1, segment 2, etc. Those names are terrible and really don't tell you anything more than their position in the list, but realistically you'd probably need to check where a section sat in a particular segment anyway, rather than just knowing that it was in there somewhere. Gabe On Mon, Oct 21, 2019 at 8:27 AM Giacomo Travaglini <[email protected]<mailto:[email protected]>> wrote: Hi Gabe, I agree we could manage to optimize this by building a sort of map with section names / addresses, where we don't do a linear scan of the section table for every segment. However my feeling is that the easiest thing to do would be to make the name tagging optional. Since the name is useful for debug purposes only (even though I don't see the name being used by any DPRINTF), I would add the name tagging thing inside a DTRACE(Loader) check. This patch implements it: https://gem5-review.googlesource.com/c/public/gem5/+/21999 Giacomo ________________________________ From: Gabe Black <[email protected]<mailto:[email protected]>> Sent: 17 October 2019 20:41 To: Giacomo Travaglini <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: Elf loading problem The reason that loop is there is that segments don't have names while sections do. Segments are regions of data that need to be loaded into memory, while sections are regions of the address space with certain properties. Segments are most meaningful to a loader, while sections are meaningful to the program itself, a linker, etc. Segments are the things we want to load, and are what ends up in the memory image. Unfortunately if there's some sort of problem with a memory image, it might not be easy to tell what the issue is if you can't tell that address blah is supposed to be read only data, or bss, or executable code, etc. The loops find out what sections are in a particular segment and name it after them, so that that relationship is preserved and can be seen later. What was happening before is that the object file classes all assumed there were exactly three sections that were loaded into memory which were the text, data, and bss. There are several problems with this. * In ELF, sections aren't loaded into memory, segments are. * There may be any number of either segments or sections. * No segment or section is formally labelled as *the* text section, etc. There may be sections which have text, data, etc., like properties, but there may not, and there may be multiple of them. If you look at the old version, you'll see how the code was attempting to scan the sections and figure out what should be the text, what should be the data, etc. It was a bit of a hack and an awkward fit which is why I got rid of it with this refactoring. I have to say, it's very odd that you have an ELF file with 3K sections and/or 3K segments. Usually there are maybe 1 or 2 segments, and maybe a dozen sections, most of which can be ignored. Assuming there's a good reason for that (if not, let's fix that), you might be able to come up with a smarter way to find which sections go with which segments. I think segments should be well behaved and not overlap with each other, but since sections just describe regions of the address space and not actual data, they can be all over the place and overlap in arbitrarily complex ways. Maybe you could loop over all the sections and build up lists of them for each segment, since if they are well behaved it would be easier to find the right segment without looping over everything? Gabe On Thu, Oct 17, 2019 at 7:00 AM Giacomo Travaglini <[email protected]<mailto:[email protected]>> wrote: Ops, 3k x 3k = 9M, not 3M ________________________________ From: gem5-dev <[email protected]<mailto:[email protected]>> on behalf of Giacomo Travaglini <[email protected]<mailto:[email protected]>> Sent: 17 October 2019 14:52 To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>>; Gabe Black <[email protected]<mailto:[email protected]>> Subject: [gem5-dev] Elf loading problem Hi Gabe, I am sending this email since I am having some elf loading problems after the patch you recently merged: https://gem5-review.googlesource.com/c/public/gem5/+/21467 I have an elf binary whose load takes a lot of time (I have set a timeout, so I can say it takes at least 30 seconds) I have debugged it and apparently we now spend a lot of time looping here. // Name segments after the sections they contain. Elf_Scn *section = elf_getscn(elf, 1); for (int sec_idx = 1; section; section = elf_getscn(elf, ++sec_idx)) { GElf_Shdr shdr; gelf_getshdr(section, &shdr); char *sec_name = elf_strptr(elf, ehdr.e_shstrndx, shdr.sh_name); if (!sec_name) { Elf_Error errorNum = (Elf_Error)elf_errno(); if (errorNum != ELF_E_NONE) { const char *errorMessage = elf_errmsg(errorNum); fatal("Error from libelf: %s.\n", errorMessage); } } if (shdr.sh_addr >= mem_start && shdr.sh_addr < mem_end) { if (name != "") name += ","; name += sec_name; } } This code loops for EVERY section in the elf, and it does that for EVERY segment. Consider that the specific example binary I have (there are more) has ~3k sections and ~3k headers, so the loader iterates 3 million times. Machine: AArch64 Version: 0x1 Entry point address: 0x0 Start of program headers: 1613144 (bytes into file) Start of section headers: 1774928 (bytes into file) Flags: 0x0 Size of this header: 64 (bytes) Size of program headers: 56 (bytes) Number of program headers: 2889 Size of section headers: 64 (bytes) Number of section headers: 2907 Section header string table index: 2905 I think this is not necessary. We should loop over sections once, annotate their address, and then start the handleLoadableSegment. I am not an elf expert, but I think part of the problems comes with the fact that we are not able to extract from the elf which sections a segment contains without looping over the entire section table. I am also wondering if it is really needed that we do all of this. As far as I can see we just do that to name the segment after the sections it contains. Is there a reason for it other than maybe debugging? Previous version of the code was avoiding it (it was doing it wrong, and this is why we never encountered that problem maybe) Please let me know your thoughts Giacomo IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ gem5-dev mailing list [email protected]<mailto:[email protected]> http://m5sim.org/mailman/listinfo/gem5-dev IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
