[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #28 from Richard Earnshaw --- (In reply to David Binderman from comment #5) > No idea. I know the gcc project is over 30 years old and it is not > feasible for me to download the entire history, it is too large. > > I have the last 18 months or so history and that's a whopping > 3.8 Gig on it's own. $ cd ~/gnusrc/gcc/master/.git $ du -sh . 1.8G. So on my machine the entire git history is just 1.8G; that's because the history is very densely packed on the server and pulling the entire history does not require an unpack-repack-send sequence. But if you download a partial history, then the git server has to unpack and then repack the required history in order to send it; that makes the process much slower and results in far more data being transmitted (the on-the-fly repack is not as dense because it would take too much time).
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #27 from Richard Earnshaw --- > ==9933==by 0x151D554: search_line_fast (lex.cc:872) This is the entry code; so the issue is with the initial alignment code (unless the buffer is smaller than 16 bytes, when we might get both under reading and overreading).
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #26 from Richard Earnshaw --- I think it's more likely that this is at the start of the buffer rather than the end, and related to rounding the address down to a 16-byte alignment. But it could also occur at the end of the buffer as well if the buffer is (nearly) full.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #25 from Mark Wielaard --- Note comment #16 which explains that valgrind seems to translate this large read into smaller chunks. Which most likely causes memcheck to flag the (last) 8 bytes read as fully invalid. See --partial-loads-ok= [default: yes] Controls how Memcheck handles 32-, 64-, 128- and 256-bit naturally aligned loads from addresses for which some bytes are addressable and others are not. When yes, such loads do not produce an address error. Instead, loaded bytes originating from illegal addresses are marked as uninitialised, and those corresponding to legal addresses are handled in the normal way. When no, loads from partially invalid addresses are treated the same as loads from completely invalid addresses: an illegal-address error is issued, and the resulting bytes are marked as initialised. It would be helpful to see if someone with arm knowledge (and valgrind VEX knowledge) can see if there is a better translation of the vld1 instruction so that it is one big read. That way memcheck at least has a chance of detecting that the part that is invalid isn't actually used. See https://sourceware.org/cgit/valgrind/tree/VEX/priv/guest_arm_toIR.c#n8383 But maybe there is no good/natural translation of these vector loads that would help memcheck see it is a valid read and only the defined bytes are used.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #24 from Richard Earnshaw --- (In reply to David Binderman from comment #22) > Is the optimization still worthwhile some 12 years later ? Almost certainly. Vector operations have become much better than they were at the time the patch went in, so it's probably even more worthwhile.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #23 from Andreas Schwab --- valgrind replaces str/mem functions with its own versions so that it can do better checking and avoid false positives.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #22 from David Binderman --- (In reply to Richard Earnshaw from comment #21) > commit e75b54a2d932929a9b2e940c5aad1ef33a86c008 > Author: Richard Earnshaw > Date: Thu Mar 22 17:54:55 2012 + > > * lex.c (search_line_fast): Provide Neon-optimized version for ARM. Is the optimization still worthwhile some 12 years later ?
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #21 from Richard Earnshaw --- FTR it was this patch that added this code. So 2012! commit e75b54a2d932929a9b2e940c5aad1ef33a86c008 Author: Richard Earnshaw Date: Thu Mar 22 17:54:55 2012 + * lex.c (search_line_fast): Provide Neon-optimized version for ARM. From-SVN: r185702 diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog index 97177e89916..133620b3b70 100644 --- a/libcpp/ChangeLog +++ b/libcpp/ChangeLog @@ -1,3 +1,7 @@ +2012-03-22 Richard Earnshaw + + * lex.c (search_line_fast): Provide Neon-optimized version for ARM. +
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 Richard Earnshaw changed: What|Removed |Added CC||rearnsha at gcc dot gnu.org --- Comment #20 from Richard Earnshaw --- (In reply to Andrew Pinski from comment #9) > This is almost definitely a valgrind issue. > We start with: > /* Align the source pointer. */ > misalign = (uintptr_t)s & 15; > p = (const uint8_t *)((uintptr_t)s & -16); > data = vld1q_u8 (p); > > > Which all other targets do too. > > Basically this is how you realign the pointer and if don't depend on the > bytes that is not in the original pointer, then this is valid. > > Does it work correctly without valgrind? Yes, for the first fetch, we align down to a 16-byte boundary and fetch the full 16 bytes. We then mask off the bytes that are before the real start of the buffer so that they cannot affect the result. So the code is safe, but valgrind has no real way of knowing this. Tricks like this wouldn't work with capability pointers, but we're not concerned about that here; even MTE (on aarch64) would be ok because the alignment used matches the tag granule size. So I'm pretty sure this is a false positive. But perhaps we should just disable the vectorized scanning when valgrind checking is enabled. Note that glibc implementations of str* functions can perform a similar trick, but perhaps valgrind has special knowledge of such cases.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #19 from Mark Wielaard --- (In reply to David Binderman from comment #18) > (In reply to Mark Wielaard from comment #17) > > I am surprised valgrind memcheck doesn't produce more output, normally it > > would tell you why & where it found the address invalid. > > The valgrind output I gave originally looks to be in the usual valgrind > format to me. > Perhaps you are assuming some other debugging tool like asan or ubsan ? Normally valgrind adds something like: Address 0xaa is x bytes inside a block of size y free'd please a stacktrace where that block was allocated/freed In this case I would expect to say something like that the address that is being access is after a block. Assuming it is indeed not accessible. The read of 4 bytes is interesting, it seems to mean that valgrind decided to chop up this read into smaller blocks. > > I assume somehow > > valgrind memcheck believes it is reading past the end of a data buffer, > > while the code assumes this is fine because it will mask off the bits it > > won't use. > > valgrind doesn't normally produce an error for copying around un-initialised > bytes. > > However, it will produce an error if those bytes are used in a decision > like an if statement. Or it will produce an error if it is an unaddressible location. Which seems to be the case here. I would try to figure out which address exactly it is, what the exact arm/neon (?) instruction it is that is being executed. How many bytes it is supposed to read and if that many bytes are actually available. If this is an overread then you might try --partial-loads-ok=yes (although that should be the default these days). If that doesn't work then valgrind might have chopped up the read into smaller blocks, so memcheck cannot see that it is a larger load and the backend (VEX/priv/guest_arm_toIR.c) might have to be adjusted.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #18 from David Binderman --- (In reply to Mark Wielaard from comment #17) > I am surprised valgrind memcheck doesn't produce more output, normally it > would tell you why & where it found the address invalid. The valgrind output I gave originally looks to be in the usual valgrind format to me. Perhaps you are assuming some other debugging tool like asan or ubsan ? > I assume somehow > valgrind memcheck believes it is reading past the end of a data buffer, > while the code assumes this is fine because it will mask off the bits it > won't use. valgrind doesn't normally produce an error for copying around un-initialised bytes. However, it will produce an error if those bytes are used in a decision like an if statement. Your unconfirmed assumption agrees with mine, though.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #17 from Mark Wielaard --- I am surprised valgrind memcheck doesn't produce more output, normally it would tell you why & where it found the address invalid. I assume somehow valgrind memcheck believes it is reading past the end of a data buffer, while the code assumes this is fine because it will mask off the bits it won't use.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #16 from Jonathan Wakely --- Not sheer fluke, it was the same ^d4ba3b369c commit both times, because that was the oldest commit in your clone.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #15 from David Binderman --- (In reply to Jonathan Wakely from comment #12) > Because otherwise I'm going to get blamed for every bug older than > 2022-11-01! > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111270#c1 My apologies for this. AFAIK sheer fluke git produced your name twice. I have extended my local history out to 20210101, nearly three years. That should reduce the size of the problem and I now know where to get full history.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #14 from David Binderman --- (In reply to Andrew Pinski from comment #9) > Does it work correctly without valgrind? Yes. No one has yet attempted to reproduce my results. vld1q_u8 is a 128 bit ARM hardware instruction. I assume that the requirements for this instruction are not being met in some way by some data. Maybe a short string ?
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 Richard Biener changed: What|Removed |Added Target||arm --- Comment #13 from Richard Biener --- Yeah, I think that's an artifact of using valgrind. But I can't see how valgrind could be fixed here apart from trying to special-case a bigger instruction pattern.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #12 from Jonathan Wakely --- Because otherwise I'm going to get blamed for every bug older than 2022-11-01! https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111270#c1
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #11 from Jonathan Wakely --- (In reply to David Binderman from comment #5) > I have the last 18 months or so history and that's a whopping > 3.8 Gig on it's own. I have a full clone with all history and it's only 3.3g, I'm not sure what you have there. (In reply to David Binderman from comment #6) > I think it would be a reasonable git enhancement if it could > handle the last (year, 3 years, 5 years, 10 years) of commits > without having to download 30+ years of commits. Git can already do that fine. You just told it to create a shallow clone with a year of history, so of course it can't tell you about anything older than that. If you increase the depth to 3 years, or 10 years, then it will be able to tell you about that history. Anyway, if you don't want a full clone that's fine - just realise the limitations of that and use something different to find who wrote the code: https://gcc.gnu.org/git/?p=gcc.git;a=blame;f=libcpp/lex.cc;h=273f6b263c8e0f061d643cae9d7f56a22379c773;hb=HEAD
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #10 from Jonathan Wakely --- (In reply to David Binderman from comment #5) > (In reply to Jonathan Wakely from comment #3) > > I don't recognise that code, are you sure I wrote it? d4ba3b369c did not > > touch that code. > > No idea. git blame is known to lie from time to time. I am no > expert at it. It never lies. Here it's telling you the lines in question are present in the oldest commit in your repo. It's not git's fault that your history stops at that commit. You can use https://gcc.gnu.org/git or https://github.com/gcc-mirror/gcc to see the full history (and check blame for that file).
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 Andrew Pinski changed: What|Removed |Added Status|UNCONFIRMED |WAITING Ever confirmed|0 |1 Last reconfirmed||2023-12-17 --- Comment #9 from Andrew Pinski --- This is almost definitely a valgrind issue. We start with: /* Align the source pointer. */ misalign = (uintptr_t)s & 15; p = (const uint8_t *)((uintptr_t)s & -16); data = vld1q_u8 (p); Which all other targets do too. Basically this is how you realign the pointer and if don't depend on the bytes that is not in the original pointer, then this is valid. Does it work correctly without valgrind?
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #8 from Andrew Pinski --- e75b54a2d932 libcpp/lex.c (Richard Earnshaw 2012-03-22 17:54:55 + 869) /* Align the source pointer. */
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #7 from David Binderman --- I tried: $ git fetch --shallow-since=1/1/2021 and the blame still has ^ on the front of it. ^abca670596 libcpp/lex.c (Ian Lance Taylor 2020-12-31 11:23:30 -0800 869) /* Align the source pointer. */ ^abca670596 libcpp/lex.c (Ian Lance Taylor 2020-12-31 11:23:30 -0800 870) misalign = (uintptr_t)s & 15; ^abca670596 libcpp/lex.c (Ian Lance Taylor 2020-12-31 11:23:30 -0800 871) p = (const uint8_t *)((uintptr_t)s & -16); ^abca670596 libcpp/lex.c (Ian Lance Taylor 2020-12-31 11:23:30 -0800 872) data = vld1q_u8 (p); IMHO, most ARM bugs seem to land in the in-tray of Richard Sandiford. Maybe Richard might recognise the code ?
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #6 from David Binderman --- (In reply to Jonathan Wakely from comment #4) > That's what the ^ on the commit suggests is happening. Righto. > You can fix your history with: > git fetch --unshallow trunk.year $ git fetch --unshallow remote: Enumerating objects: 1072084 ^C trunk.year $ It is not practical for me to download more than a million objects. I think it would be a reasonable git enhancement if it could handle the last (year, 3 years, 5 years, 10 years) of commits without having to download 30+ years of commits.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #5 from David Binderman --- (In reply to Jonathan Wakely from comment #3) > I don't recognise that code, are you sure I wrote it? d4ba3b369c did not > touch that code. No idea. git blame is known to lie from time to time. I am no expert at it. > Do you have a shallow clone, which happens to have my d4ba3b369c as the > oldest commit? No idea. I know the gcc project is over 30 years old and it is not feasible for me to download the entire history, it is too large. I have the last 18 months or so history and that's a whopping 3.8 Gig on it's own.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #4 from Jonathan Wakely --- That's what the ^ on the commit suggests is happening. You can fix your history with: git fetch --unshallow
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #3 from Jonathan Wakely --- I don't recognise that code, are you sure I wrote it? d4ba3b369c did not touch that code. Do you have a shallow clone, which happens to have my d4ba3b369c as the oldest commit?
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 David Binderman changed: What|Removed |Added CC||redi at gcc dot gnu.org --- Comment #2 from David Binderman --- The source code in lex.cc is /* Align the source pointer. */ misalign = (uintptr_t)s & 15; p = (const uint8_t *)((uintptr_t)s & -16); data = vld1q_u8 (p); git blame has ^d4ba3b369c (Jonathan Wakely 2022-11-01 09:48:41 + 869) /* Align the source pointer. */ ^d4ba3b369c (Jonathan Wakely 2022-11-01 09:48:41 + 870) misalign = (uintptr_t)s & 15; ^d4ba3b369c (Jonathan Wakely 2022-11-01 09:48:41 + 871) p = (const uint8_t *)((uintptr_t)s & -16); ^d4ba3b369c (Jonathan Wakely 2022-11-01 09:48:41 + 872) data = vld1q_u8 (p); Adding author of code for their opinion.
[Bug target/113045] armv7l-unknown-linux-gnueabihf: valgrind error during build of libcc1
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113045 --- Comment #1 from Andrew Pinski --- This could also be a valgrind issue ...
