There are some problems with that patch which I described in the review. To circle back, why does your ELF file have such a crazy number of segments and sections? That seems highly unusual, and it would be nice not to have to go to extraordinary lengths to support such an atypical input.
I would say if it really is prohibitive to figure out names based on sections, to just call the segments segment 1, segment 2, etc. Those names are terrible and really don't tell you anything more than their position in the list, but realistically you'd probably need to check where a section sat in a particular segment anyway, rather than just knowing that it was in there somewhere. Gabe On Mon, Oct 21, 2019 at 8:27 AM Giacomo Travaglini < [email protected]> wrote: > Hi Gabe, > > I agree we could manage to optimize this by building a sort of map with > section names / addresses, where we don't do a linear scan of the section > table > for every segment. > > However my feeling is that the easiest thing to do would be to make the > name tagging optional. > Since the name is useful for debug purposes only (even though I don't see > the name being used by any DPRINTF), I would add the name tagging thing > inside a DTRACE(Loader) check. > > This patch implements it: > > https://gem5-review.googlesource.com/c/public/gem5/+/21999 > > Giacomo > > > > > ------------------------------ > *From:* Gabe Black <[email protected]> > *Sent:* 17 October 2019 20:41 > *To:* Giacomo Travaglini <[email protected]> > *Cc:* [email protected] <[email protected]> > *Subject:* Re: Elf loading problem > > The reason that loop is there is that segments don't have names while > sections do. Segments are regions of data that need to be loaded into > memory, while sections are regions of the address space with certain > properties. Segments are most meaningful to a loader, while sections are > meaningful to the program itself, a linker, etc. > > Segments are the things we want to load, and are what ends up in the > memory image. Unfortunately if there's some sort of problem with a memory > image, it might not be easy to tell what the issue is if you can't tell > that address blah is supposed to be read only data, or bss, or executable > code, etc. The loops find out what sections are in a particular segment and > name it after them, so that that relationship is preserved and can be seen > later. > > What was happening before is that the object file classes all assumed > there were exactly three sections that were loaded into memory which were > the text, data, and bss. There are several problems with this. > * In ELF, sections aren't loaded into memory, segments are. > * There may be any number of either segments or sections. > * No segment or section is formally labelled as *the* text section, etc. > There may be sections which have text, data, etc., like properties, but > there may not, and there may be multiple of them. > > If you look at the old version, you'll see how the code was attempting to > scan the sections and figure out what should be the text, what should be > the data, etc. It was a bit of a hack and an awkward fit which is why I got > rid of it with this refactoring. > > I have to say, it's very odd that you have an ELF file with 3K sections > and/or 3K segments. Usually there are maybe 1 or 2 segments, and maybe a > dozen sections, most of which can be ignored. Assuming there's a good > reason for that (if not, let's fix that), you might be able to come up with > a smarter way to find which sections go with which segments. I think > segments should be well behaved and not overlap with each other, but since > sections just describe regions of the address space and not actual data, > they can be all over the place and overlap in arbitrarily complex ways. > > Maybe you could loop over all the sections and build up lists of them for > each segment, since if they are well behaved it would be easier to find the > right segment without looping over everything? > > Gabe > > On Thu, Oct 17, 2019 at 7:00 AM Giacomo Travaglini < > [email protected]> wrote: > > Ops, 3k x 3k = 9M, not 3M > ------------------------------ > *From:* gem5-dev <[email protected]> on behalf of Giacomo > Travaglini <[email protected]> > *Sent:* 17 October 2019 14:52 > *To:* [email protected] <[email protected]>; Gabe Black < > [email protected]> > *Subject:* [gem5-dev] Elf loading problem > > Hi Gabe, > > I am sending this email since I am having some elf loading problems after > the patch you recently merged: > > https://gem5-review.googlesource.com/c/public/gem5/+/21467 > > I have an elf binary whose load takes a lot of time (I have set a timeout, > so I can say it takes at least 30 seconds) > > I have debugged it and apparently we now spend a lot of time looping here. > > // Name segments after the sections they contain. > Elf_Scn *section = elf_getscn(elf, 1); > for (int sec_idx = 1; section; section = elf_getscn(elf, ++sec_idx)) { > GElf_Shdr shdr; > gelf_getshdr(section, &shdr); > char *sec_name = elf_strptr(elf, ehdr.e_shstrndx, shdr.sh_name); > if (!sec_name) { > Elf_Error errorNum = (Elf_Error)elf_errno(); > if (errorNum != ELF_E_NONE) { > const char *errorMessage = elf_errmsg(errorNum); > fatal("Error from libelf: %s.\n", errorMessage); > } > } > if (shdr.sh_addr >= mem_start && shdr.sh_addr < mem_end) { > if (name != "") > name += ","; > name += sec_name; > } > } > > > This code loops for EVERY section in the elf, and it does that for EVERY > segment. > Consider that the specific example binary I have (there are more) has ~3k > sections and ~3k headers, > so the loader iterates 3 million times. > > Machine: AArch64 > Version: 0x1 > Entry point address: 0x0 > Start of program headers: 1613144 (bytes into file) > Start of section headers: 1774928 (bytes into file) > Flags: 0x0 > Size of this header: 64 (bytes) > Size of program headers: 56 (bytes) > Number of program headers: 2889 > Size of section headers: 64 (bytes) > Number of section headers: 2907 > Section header string table index: 2905 > > > I think this is not necessary. We should loop over sections once, annotate > their address, and then start the handleLoadableSegment. > I am not an elf expert, but I think part of the problems comes with the > fact that we are not able to extract from the elf which sections a segment > contains without looping over the entire section table. > > I am also wondering if it is really needed that we do all of this. As far > as I can see we just do that to name the segment after the sections it > contains. Is there a reason for it other than maybe debugging? Previous > version of the code was avoiding it (it was doing it wrong, and this is why > we never encountered that problem maybe) > > Please let me know your thoughts > > Giacomo > > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > _______________________________________________ > gem5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/gem5-dev > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
