[Bug sanitizer/113430] [11/12/13 only] Trivial program segfaults intermittently with ASAN with large CONFIG_ARCH_MMAP_RND_BITS in kernel configuration

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113430

--- Comment #11 from Dimitrij Mijoski  ---
(In reply to Sam James from comment #10)
> I don't plan on pursuing it myself, leaving it to someone else, as I can't
> reproduce on my main workstation and I don't want to faff w/ kernel config.

You should be able to modify the kernel parameter at runtime by running:

sudo sysctl vm.mmap_rnd_bits=32

That should be enough to reproduce the issue. The fix is to cherry-pick the
changes to asan_allocator.h but also to lsan_allocator.h from this patch
r14-263-gd53b3d94aaf211ffb2159614f5aaaf03ceb861cc. You missed lsan_allocator.h
in your patch.

[Bug sanitizer/113430] Trivial program segfaults intermittently with ASAN with large CONFIG_ARCH_MMAP_RND_BITS in kernel configuration

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113430

Dimitrij Mijoski  changed:

   What|Removed |Added

 CC||dmjpp at hotmail dot com

--- Comment #8 from Dimitrij Mijoski  ---
This bug manifested at large on Github Actions CI/CI system in the last few
days most likely because Ubuntu's kernel also got updated to use 32 random
bits. Here is the bug report
https://github.com/actions/runner-images/issues/9491 . It would be a good idea
to backport the fix.

[Bug libstdc++/108976] codecvt for Unicode allows surrogate code points

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108976

--- Comment #11 from Dimitrij Mijoski  ---
(In reply to Jonathan Wakely from comment #10)
> I think it would be good to backport it, what do you think?

I don't really have strong need. Maybe porting only to v13 as that is pretty
straightforward (simple cherry-picking). Porting to v12 and v11 will require
porting other patches first.

[Bug libstdc++/108976] codecvt for Unicode allows surrogate code points

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108976

--- Comment #9 from Dimitrij Mijoski  ---
I believe this bug report should closed as resolved. Are there maybe plans for
back-porting?

[Bug libstdc++/108976] codecvt for Unicode allows surrogate code points

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108976

--- Comment #7 from Dimitrij Mijoski  ---
I put a second version of the patch
https://gcc.gnu.org/pipermail/libstdc++/2023-March/055667.html about a month
ago.

[Bug libstdc++/108976] codecvt for Unicode allows surrogate code points

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108976

--- Comment #6 from Dimitrij Mijoski  ---
I sent a single patch to the mailing list with a good detailed commit message.
I think that is better than multiple patches.

[Bug libstdc++/108976] codecvt for Unicode allows surrogate code points

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108976

--- Comment #4 from Dimitrij Mijoski  ---
I have fixed this, added large testsute and discovered another bug in
codecvt_utf16 when the input [from, from_end) contains odd number of bytes.
Error was returned instead of partial.

Here are the changes in 8 commits
https://github.com/gcc-mirror/gcc/compare/master...dimztimz:gcc:codecvt (read
from top to bottom). Do you want everything squashed in one commit or maybe
some other combination?

[Bug libstdc++/108976] codecvt for Unicode allows surrogate code points

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108976

--- Comment #2 from Dimitrij Mijoski  ---
(In reply to Dimitrij Mijoski from comment #0)
> Those that read from UCS-2 seem to me like they properly report the error.
> Reading from UTF-16 can not have this bug by definition. From what I
> checked, the functions for reading UTF-16 properly treat unpaired surrogate
> code units as error.

Seems like the conversion from UCS-2 to UTF-16BE/LE is also affected. This
conversions is called via codecvt_utf16::out(). See line
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libstdc%2B%2B-v3/src/c%2B%2B11/codecvt.cc;h=02f05752de84139a7eb7c3d40946b61f4c0334cf;hb=HEAD#l656
it only checks for high surrogate but should also check for low.

[Bug libstdc++/108976] New: codecvt for Unicode allows surrogate code points

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108976

Bug ID: 108976
   Summary: codecvt for Unicode allows surrogate code points
   Product: gcc
   Version: 13.0
Status: UNCONFIRMED
  Severity: normal
  Priority: P3
 Component: libstdc++
  Assignee: unassigned at gcc dot gnu.org
  Reporter: dmjpp at hotmail dot com
  Target Milestone: ---

Text in valid Unicode should never contain surrogate code POINTS. Those are
only allowed in UTF-16, but only as code UNITS and must be properly paired.

UTF-8 text in its strictest form must not contain surrogates but in a slightly
relaxed form surrogates can be easily encoded as 3-byte sequences. Same can be
said for UTF-32 and UCS-2. Only UTF-16 is immune to the error of surrogate code
POINT (they are treated as UNITS).

Codecvts in libstdc++ currently allow surrogate code points in some cases. Here
is a minimal reproduction (asserts are the correct behavior):

#include 
#include 

void u32()
{
using namespace std;
auto& f = use_facet>(locale::classic());

char u8str[] = "\uC800\uCBFF\uCC00\uCFFF";
u8str[0] = u8str[3] = u8str[6] = u8str[9] = 0xED; // turn the C into D.
// now the string is D800, DBFF, DC00 and DFFF encoded in relaxed UTF-8
// that allows surrogate code points.
char32_t u32str[] = {0xD800, 0xDBFF, 0xDC00, 0xDFFF, 0};

char32_t u32out[1];
const char* from_next;
char32_t* to_next;
mbstate_t st = {};
auto res = f.in(st, u8str, u8str+3, from_next, u32out, u32out+1, to_next);
assert(res == f.error);
assert(from_next == u8str);
assert(to_next == u32out);

st = {};
auto l = f.length(st, u8str, u8str+3, 1);
assert(l == 0);

char u8out[3];
const char32_t* from_next2;
char* to_next2;
st = {};
res = f.out(st, u32str, u32str+1, from_next2, u8out, u8out+3, to_next2);
assert(res == f.error);
assert(from_next2 == u32str);
assert(to_next2 == u8out);
}
void u16()
{
using namespace std;
auto& f = use_facet>(locale::classic());

char u8str[] = "\uC800\uCBFF\uCC00\uCFFF";
u8str[0] = u8str[3] = u8str[6] = u8str[9] = 0xED; // turn the C into D.
// now the string is D800, DBFF, DC00 and DFFF encoded in relaxed UTF-8
// that allows surrogates.

char16_t u16out[1];
const char* from_next;
char16_t* to_next;
mbstate_t st = {};
auto res = f.in(st, u8str, u8str+3, from_next, u16out, u16out+1, to_next);
assert(res == f.error);
assert(from_next == u8str);
assert(to_next == u16out);

st = {};
auto l = f.length(st, u8str, u8str+3, 1);
assert(l == 0);
}
int main()
{
u32();
u16();
}

>From reading the file codecvt.cc the following conversions have the bug:

- From UTF-8 to any other encoding.
- From UTF-32/UCS-4 to any other encoding.

Those that read from UCS-2 seem to me like they properly report the error.
Reading from UTF-16 can not have this bug by definition. From what I checked,
the functions for reading UTF-16 properly treat unpaired surrogate code units
as error.

[Bug c++/106656] [C++23] P2513 - char8_t Compatibility and Portability Fixes

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106656

Dimitrij Mijoski  changed:

   What|Removed |Added

 CC||dmjpp at hotmail dot com

--- Comment #3 from Dimitrij Mijoski  ---
The documentation for CLI flag -fchar8_t should be updated
https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Dialect-Options.html .

Is there any chance that this fix gets backported because it is treated as
defect report to C++20?

[Bug libstdc++/98466] Debug Mode iterators for unordered containers do not implement N3644

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98466

--- Comment #3 from Dimitrij Mijoski  ---
(In reply to Jonathan Wakely from comment #2)
> This was already fixed on master by r11-6682
> 05a30af3f237984b4dcf1dbbc17fdac583c46506

Yes, that patch mostly fixes bug 70303, too. With that patch, the asserts
presented in bug 70303 pass for vector::iterator but not for
deque::iterator.

[Bug libstdc++/98466] Debug Mode iterators for unordered containers do not implement N3644

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98466

Dimitrij Mijoski  changed:

   What|Removed |Added

 CC||dmjpp at hotmail dot com

--- Comment #1 from Dimitrij Mijoski  ---
This bug looks like a duplicate of bug 70303. The asserts presented there
should be used on random-access iterators (vector, deque) to test if N3644 is
implement.

[Bug libstdc++/86419] codecvt::in() and out() incorrectly return ok in some cases.

[dmjpp at hotmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86419

--- Comment #12 from Dimitrij Mijoski  ---
Hello Jonathan. I posted a patch for this bug which I hope you'll find it
useful once you start working on this.
https://gcc.gnu.org/pipermail/libstdc++/2020-September/051073.html